单帧图像超分辨率重建的深度神经网络综述
|
摘要
随着硬件计算能力的显著提升,深度神经网络广泛应用于计算机视觉和图像处理的各个领域,获得了突出成果,受这种方法启发,单帧图像超分辨率重建(Super-resolution Reconstruction, SR)也引入深度学习思想,并且重建效果远远超越传统算法,成为研究的热点并迅速成为主流技术。将对深度神经网络的单帧图像超分辨率重建技术分为两类(基于传统深度神经网络的单帧图像超分辨率重建和基于生成对抗网络的单帧图像超分辨率重建和)进行阐述,以此为基础,对单帧图像超分辨率技术的发展趋势进行展望。
With the significant improvement of hardware computing power, deep neural networks are widely used in various fields of computer vision and image processing, and have achieved outstanding results. Inspired by this, the field of single image Super-resolution Reconstruction(SR)is also introduced deep learning ideas, and performance beyond traditional algorithms, become a research hotspot and become mainstream technology. In this paper, the existing image super-resolution reconstruction techniques based on deep neural networks are divided into two categories(traditional deep neural single image super-resolution reconstruction and generation adversarial networks for single image super-resolution reconstruction) for description, Based on this, we will look into the development of image super-resolution technology.
With the significant improvement of hardware computing power, deep neural networks are widely used in various fields of computer vision and image processing, and have achieved outstanding results. Inspired by this, the field of single image Super-resolution Reconstruction(SR)is also introduced deep learning ideas, and performance beyond traditional algorithms, become a research hotspot and become mainstream technology. In this paper, the existing image super-resolution reconstruction techniques based on deep neural networks are divided into two categories(traditional deep neural single image super-resolution reconstruction and generation adversarial networks for single image super-resolution reconstruction) for description, Based on this, we will look into the development of image super-resolution technology.
引文
[1]李现国,孙叶美,杨彦利,苗长云.基于中间层监督卷积神经网络的图像超分辨率重建[J].中国图象图形学报,2018,23(7):0984-0993.
[2]Dong C,Loy CC,He K,Tang X.Image Super-Resolution Using Deep Convolutional Networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
[3]Dong C,Loy CC,Tang X.Accelerating the super-resolution convolutional neural network[C]//European Conference on Computer Vision,Springer,2016,391-407.
[4]Shi W,Caballero J,Huszár F,et al.Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network[C]//IEEE Conference on Computer Vision and Pattern.
[5]Kaiming He,Xiangyu Zhang,Shaoqing Ren,and Jian Sun.Deep Residual Learning for Image Recognition[C]//IEEEConference on Computer Vision and Pattern Recognition(CVPR),2016
[6]Mao X J,Shen C,Yang Y B.Image Restoration Using Convolutional Auto-encoders with Symmetric Skip Connections[C]//NIPS,2016.
[7]Lai W S,Huang J B,Ahuja N,et al.Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2017.
[8]Tong Tong,Gen Li,Xiejie Liu,Qinquan Gao.Image Super-Resolution Using Dense Skip Connections[C]//ICCV,2017
[9]Y.Zhang,Y.Tian,Y.Kong,B.Zhong,and Y.Fu.Residual dense network for image super-resolution[C]//The IEEEConference on Computer Vision and Pattern Recognition(CVPR),2018.
[10]Muhammad Haris,Greg Shakhnarovich,Norimichi Ukita.Deep Back-Projection Networks For Super-Resolution[C]//The IEEE Conference on Computer Vision and Pattern Recognition(CVPR),2018.
[11]B.Lim,S.Son,H.Kim,S.Nah,and K.M.Lee,“Enhanced deep residual networks for single image super-resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR)Workshops,vol.1,no.2,2017,p.3.
[12]J.Johnson,A.Alahi,and F.Li.Perceptual losses for realtime style transfer and super-resolution.In European Conference on Computer Vision(ECCV),pages 694-711.Springer,2016.
[13]Yulun Zhang,Kunpeng Li,Kai Li,Lichen Wang,Bineng Zhong,Yun Fu.Image Super-Resolution Using Very Deep Residual Channel Attention Ne Twork-s.ECCV,2018.
[14]Kim J,Lee J K,Lee K M.Deeply-Recursive Convolutional Network for Image Super-Resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2016:1637-1645.
[15]Ying Tai,Jian Yang,and Xiaoming Liu.Image Super-Resolution via Deep Recursive Residual Network[C]//IEEEConference on Computer Vision and Pattern Recognition.IEEE Computer Society,2017.
[16]Christian Ledig,Lucas Theis,Ferenc Husz′ar,Jose Caballero,Andrew Aitken,Alykhan Tejani,Johannes Totz,Zehan Wang,Wenzhe Shi.Photo-Realistic Single Image SuperResolution Using a Generative Adversarial Network[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2017.
[17]Sajjadi,M.S.M.and Sch?lkopf,B.and Hirsch,M.EnhanceNet:Single Image Super-Resolution through Automated Texture Synthesis[C]//IEEE International Conference on Computer Vision.2017:4491--4500.
[18]R.Mechrez,I.Talmi,F.Shama,and L.Zelnik-Manor.Learning tomaintain natural image statistics[J/OL].arXiv:1803.04626,2018.
[19]Wang X,Yu K,Wu S,et al.ESRGAN:Enhanced Super-Resolution Generative Adversarial Networks[C].//European Conference on Computer Vision(ECCV)workshops,2018.
[20]Seong-Jin Park,Hyeongseok Son,Sunghyun Cho,Ki-Sang Hong,Seungyong Lee.SRFeat:Single Image Super-Resolution with Feature Discrimination[C]//The European Conference on Computer Vision(ECCV),2018,pp.439-455
[2]Dong C,Loy CC,He K,Tang X.Image Super-Resolution Using Deep Convolutional Networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
[3]Dong C,Loy CC,Tang X.Accelerating the super-resolution convolutional neural network[C]//European Conference on Computer Vision,Springer,2016,391-407.
[4]Shi W,Caballero J,Huszár F,et al.Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network[C]//IEEE Conference on Computer Vision and Pattern.
[5]Kaiming He,Xiangyu Zhang,Shaoqing Ren,and Jian Sun.Deep Residual Learning for Image Recognition[C]//IEEEConference on Computer Vision and Pattern Recognition(CVPR),2016
[6]Mao X J,Shen C,Yang Y B.Image Restoration Using Convolutional Auto-encoders with Symmetric Skip Connections[C]//NIPS,2016.
[7]Lai W S,Huang J B,Ahuja N,et al.Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2017.
[8]Tong Tong,Gen Li,Xiejie Liu,Qinquan Gao.Image Super-Resolution Using Dense Skip Connections[C]//ICCV,2017
[9]Y.Zhang,Y.Tian,Y.Kong,B.Zhong,and Y.Fu.Residual dense network for image super-resolution[C]//The IEEEConference on Computer Vision and Pattern Recognition(CVPR),2018.
[10]Muhammad Haris,Greg Shakhnarovich,Norimichi Ukita.Deep Back-Projection Networks For Super-Resolution[C]//The IEEE Conference on Computer Vision and Pattern Recognition(CVPR),2018.
[11]B.Lim,S.Son,H.Kim,S.Nah,and K.M.Lee,“Enhanced deep residual networks for single image super-resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR)Workshops,vol.1,no.2,2017,p.3.
[12]J.Johnson,A.Alahi,and F.Li.Perceptual losses for realtime style transfer and super-resolution.In European Conference on Computer Vision(ECCV),pages 694-711.Springer,2016.
[13]Yulun Zhang,Kunpeng Li,Kai Li,Lichen Wang,Bineng Zhong,Yun Fu.Image Super-Resolution Using Very Deep Residual Channel Attention Ne Twork-s.ECCV,2018.
[14]Kim J,Lee J K,Lee K M.Deeply-Recursive Convolutional Network for Image Super-Resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2016:1637-1645.
[15]Ying Tai,Jian Yang,and Xiaoming Liu.Image Super-Resolution via Deep Recursive Residual Network[C]//IEEEConference on Computer Vision and Pattern Recognition.IEEE Computer Society,2017.
[16]Christian Ledig,Lucas Theis,Ferenc Husz′ar,Jose Caballero,Andrew Aitken,Alykhan Tejani,Johannes Totz,Zehan Wang,Wenzhe Shi.Photo-Realistic Single Image SuperResolution Using a Generative Adversarial Network[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2017.
[17]Sajjadi,M.S.M.and Sch?lkopf,B.and Hirsch,M.EnhanceNet:Single Image Super-Resolution through Automated Texture Synthesis[C]//IEEE International Conference on Computer Vision.2017:4491--4500.
[18]R.Mechrez,I.Talmi,F.Shama,and L.Zelnik-Manor.Learning tomaintain natural image statistics[J/OL].arXiv:1803.04626,2018.
[19]Wang X,Yu K,Wu S,et al.ESRGAN:Enhanced Super-Resolution Generative Adversarial Networks[C].//European Conference on Computer Vision(ECCV)workshops,2018.
[20]Seong-Jin Park,Hyeongseok Son,Sunghyun Cho,Ki-Sang Hong,Seungyong Lee.SRFeat:Single Image Super-Resolution with Feature Discrimination[C]//The European Conference on Computer Vision(ECCV),2018,pp.439-455