基于改进的深度卷积神经网络的单图像超分辨率重建
|
摘要
为解决现有的超分辨率模型不能很好的恢复图像的纹理细节和模型训练困难等问题,结合现有的残差网络和GoogleNet中的Inception模块对其进行改进。通过将5×5的卷积核替换为两个级联的3×3的卷积核、使用LeakyReLU作为激活函数和删除池化层等方法对原始的Inception模块进行改进,然后在模型中多次级联改进后的Inception模块。实验结果表明,与双三次插值算法、SRCNN和VDSR算法相比,改进后的模型能获得更高的峰值信噪比(PSNR)和结构相似性指数(SSIM),并且在视觉效果上也有明显的改善。
In order to solve the problem that the existing super-resolution model can not restore the texture details of the image and the difficulty of model training,the existing model is improved by combining the existing residual network and the Inception module in GoogleNet.The original Inception module is improved by replacing the 5×5 convolution kernel with two cascaded 3×3 convolution kernels,using LeakyReLU as the activation function,and deleting the pooling layer,and then improved Inception module is cascaded multiple times in the model.The experimental results show that compared with the bicubic interpolation,SRCNN and VDSR algorithm,the improved model can obtain higher peak signal-to-noise ratio(PSNR)and structural similarity(SSIM),and also has obvious improvement in visual effects.
In order to solve the problem that the existing super-resolution model can not restore the texture details of the image and the difficulty of model training,the existing model is improved by combining the existing residual network and the Inception module in GoogleNet.The original Inception module is improved by replacing the 5×5 convolution kernel with two cascaded 3×3 convolution kernels,using LeakyReLU as the activation function,and deleting the pooling layer,and then improved Inception module is cascaded multiple times in the model.The experimental results show that compared with the bicubic interpolation,SRCNN and VDSR algorithm,the improved model can obtain higher peak signal-to-noise ratio(PSNR)and structural similarity(SSIM),and also has obvious improvement in visual effects.
引文
[1]Keys R.Cubic convolution interpolation for digital image processing[J].IEEE transactions on acoustics,speech,and signal processing,1981,29(6):1153-1160.
[2]Fattal R.Image upsampling via imposed edge statistics[C]//ACM transactions on graphics(TOG).ACM,2007,26(3):95-103.
[3]Irani M,Peleg S.Improving resolution by image registration[J].CVGIP:Graphical models and image processing,1991,53(3):231-239.
[4]Stark H,Oskoui P.High-resolution image recovery from image-plane arrays,using convex projections[J].JOSA A,1989,6(11):1715-1726.
[5]Schultz R R,Stevenson R L.Improved definition video frame enhancement[C]//Acoustics,Speech,and Signal Processing,1995.IC-ASSP-95.,1995International Conference on.IEEE,1995,4:2169-2172.
[6]Freeman W T,Pasztor E C,Carmichael O T.Learning low-level vision[J].International journal of computer vision,2000,40(1):25-47.
[7]Gao X,Zhang K,Tao D,et al.Image super-resolution with sparse neighbor embedding[J].IEEE Transactions on Image Processing,2012,21(7):3194-3205.
[8]Yang J,Wright J,Huang T S,et al.Image super-resolution via sparse representation[J].IEEE transactions on image processing,2010,19(11):2861-2873.
[9]Dong C,Loy C C,He K,et al.Image super-resolution using deep convolutional networks[J].IEEE transactions on pattern analysis and machine intelligence,2016,38(2):295-307.
[10]肖进胜,刘恩雨,朱力,等.改进的基于卷积神经网络的图像超分辨率算法[J].光学学报,2017,37(3):96-104.
[11]郭晓,谭文安.基于级联深度卷积神经网络的高性能图像超分辨率重构[J].计算机应用,2017,37(11):3124-3127.
[12]Kim J,Kwon Lee J,Mu Lee K.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEEconference on computer vision and pattern recognition.2016:1646-1654.
[13]Szegedy C,Liu W,Jia Y,et al.Going deeper with convolutions[C]//Proceedings of the IEEE conference on computer vision and pattern recognition.2015:1-9.
[14]Nair V,Hinton G E.Rectified linear units improve restricted boltzmann machines[C]//Proceedings of the 27th international conference on machine learning(ICML-10).2010:807-814.
[15]Maas A L,Hannun A Y,Ng A Y.Rectifier nonlinearities improve neural network acoustic models[C]//Proc.icml.2013,30(1):3.
[16]Kim J,Kwon Lee J,Mu Lee K.Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of the IEEE conference on computer vision and pattern recognition.2016:1637-1645.
[17]He K,Zhang X,Ren S,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition.2016:770-778.
[18]Wang Z,Bovik A C,Sheikh H R,et al.Image quality assessment:from error visibility to structural similarity[J].IEEE transactions on image processing,2004,13(4):600-612.
[19]Yang J,Wright J,Huang T S,et al.Image super-resolution via sparse representation[J].IEEE transactions on image processing,2010,19(11):2861-2873.
[20]Arbelaez P,Maire M,Fowlkes C,et al.Contour detection and hierarchical image segmentation[J].IEEE transactions on pattern analysis and machine intelligence,2011,33(5):898-916.
[21]Glorot X,Bengio Y.Understanding the difficulty of training deep feedforward neural networks[C]//Proceedings of the thirteenth international conference on artificial intelligence and statistics.2010:249-256.
[22]Zeyde R,Elad M,Protter M.On single image scale-up using sparse-representations[C]//International conference on curves and surfaces.Springer,Berlin,Heidelberg,2010:711-730.
[2]Fattal R.Image upsampling via imposed edge statistics[C]//ACM transactions on graphics(TOG).ACM,2007,26(3):95-103.
[3]Irani M,Peleg S.Improving resolution by image registration[J].CVGIP:Graphical models and image processing,1991,53(3):231-239.
[4]Stark H,Oskoui P.High-resolution image recovery from image-plane arrays,using convex projections[J].JOSA A,1989,6(11):1715-1726.
[5]Schultz R R,Stevenson R L.Improved definition video frame enhancement[C]//Acoustics,Speech,and Signal Processing,1995.IC-ASSP-95.,1995International Conference on.IEEE,1995,4:2169-2172.
[6]Freeman W T,Pasztor E C,Carmichael O T.Learning low-level vision[J].International journal of computer vision,2000,40(1):25-47.
[7]Gao X,Zhang K,Tao D,et al.Image super-resolution with sparse neighbor embedding[J].IEEE Transactions on Image Processing,2012,21(7):3194-3205.
[8]Yang J,Wright J,Huang T S,et al.Image super-resolution via sparse representation[J].IEEE transactions on image processing,2010,19(11):2861-2873.
[9]Dong C,Loy C C,He K,et al.Image super-resolution using deep convolutional networks[J].IEEE transactions on pattern analysis and machine intelligence,2016,38(2):295-307.
[10]肖进胜,刘恩雨,朱力,等.改进的基于卷积神经网络的图像超分辨率算法[J].光学学报,2017,37(3):96-104.
[11]郭晓,谭文安.基于级联深度卷积神经网络的高性能图像超分辨率重构[J].计算机应用,2017,37(11):3124-3127.
[12]Kim J,Kwon Lee J,Mu Lee K.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEEconference on computer vision and pattern recognition.2016:1646-1654.
[13]Szegedy C,Liu W,Jia Y,et al.Going deeper with convolutions[C]//Proceedings of the IEEE conference on computer vision and pattern recognition.2015:1-9.
[14]Nair V,Hinton G E.Rectified linear units improve restricted boltzmann machines[C]//Proceedings of the 27th international conference on machine learning(ICML-10).2010:807-814.
[15]Maas A L,Hannun A Y,Ng A Y.Rectifier nonlinearities improve neural network acoustic models[C]//Proc.icml.2013,30(1):3.
[16]Kim J,Kwon Lee J,Mu Lee K.Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of the IEEE conference on computer vision and pattern recognition.2016:1637-1645.
[17]He K,Zhang X,Ren S,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition.2016:770-778.
[18]Wang Z,Bovik A C,Sheikh H R,et al.Image quality assessment:from error visibility to structural similarity[J].IEEE transactions on image processing,2004,13(4):600-612.
[19]Yang J,Wright J,Huang T S,et al.Image super-resolution via sparse representation[J].IEEE transactions on image processing,2010,19(11):2861-2873.
[20]Arbelaez P,Maire M,Fowlkes C,et al.Contour detection and hierarchical image segmentation[J].IEEE transactions on pattern analysis and machine intelligence,2011,33(5):898-916.
[21]Glorot X,Bengio Y.Understanding the difficulty of training deep feedforward neural networks[C]//Proceedings of the thirteenth international conference on artificial intelligence and statistics.2010:249-256.
[22]Zeyde R,Elad M,Protter M.On single image scale-up using sparse-representations[C]//International conference on curves and surfaces.Springer,Berlin,Heidelberg,2010:711-730.