深度非对称跳跃连接的图像降噪方法
|
摘要
图像降噪可有效地改善图像质量,提升感官效果,也是图像特征提取与理解的前提.针对目前比较流行的卷积神经网络降噪方法中顺序连接的卷积层-反卷积层会使图像在梯度反传过程中逐渐弱化图像噪声的学习问题,提出一种深度非对称跳跃连接的方法用于图像降噪.该方法设计多组非对称跳跃连接卷积-反卷积算子,以有效学习图像细节及噪声信息,并对不同深度的卷积操作进行权重量化,以加强图像降噪及恢复;通过非对称跳跃连接可使图像噪声信息能够直接反传到对应的多个卷积层中,对梯度扩散有良好的抑制作用.采用伯克利分割数据集BSD300进行实验的结果表明,文中算法比基准方法在结构相似性(SSIM)和峰值信噪比(PSNR)2种指标上都有提升.
Image denoising can effectively improve image quality and sensory effect, and is also the premise of image feature extraction and understanding. For the current popular convolution neural network denoising methods, sequentially connected convolution-deconvolution layer will gradually weaken the image noise in the gradient back propagation process, a method of deep asymmetrical skip connection is proposed for image denoising. In this method, several asymmetrical skip convolution-deconvolution operators are designed to effectively learn image details and noise information, the weights of convolution operations with different depths are quantized to enhance image denoising and restoration. The asymmetrical skip connection can make the image noise information be transmitted directly back to the corresponding convolution layers, which has a good inhibition on gradient diffusion. Experiments on a Berkeley Segmentation Dataset BSD300 show that the proposed algorithm can improve both structural similarity(SSIM) and peak signal-to-noise ratio(PSNR) compared with the benchmark method.
Image denoising can effectively improve image quality and sensory effect, and is also the premise of image feature extraction and understanding. For the current popular convolution neural network denoising methods, sequentially connected convolution-deconvolution layer will gradually weaken the image noise in the gradient back propagation process, a method of deep asymmetrical skip connection is proposed for image denoising. In this method, several asymmetrical skip convolution-deconvolution operators are designed to effectively learn image details and noise information, the weights of convolution operations with different depths are quantized to enhance image denoising and restoration. The asymmetrical skip connection can make the image noise information be transmitted directly back to the corresponding convolution layers, which has a good inhibition on gradient diffusion. Experiments on a Berkeley Segmentation Dataset BSD300 show that the proposed algorithm can improve both structural similarity(SSIM) and peak signal-to-noise ratio(PSNR) compared with the benchmark method.
引文
[1]Milanfar P.A tour of modern image filtering:new insights and methods,both practical and theoretical[J].IEEE Signal Processing Magazine,2013,30(1):106-128
[2]Wang Z Y,Yang Y Z,Wang Z W,et al.Learning super-resolution jointly from external and internal examples[J].IEEETransactions on Image Processing,2015,24(11):4359-4371
[3]Gu S H,Zuo W M,Xie Q,et al.Convolutional sparse coding for image super-resolution[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2015:1823-1831
[4]Chatterjee P,Milanfar P.Clustering-based denoising with locally learned dictionaries[J].IEEE Transactions on Image Processing,2009,18(7):1438-1451
[5]Dabov K,Foi A,Katkovnik V,et al.Image denoising by sparse3-D transform-domain collaborative filtering[J].IEEE Transactions on Image Processing,2007,16(8):2080-2095
[6]Gu S H,Zhang L,Zuo W M,et al.Weighted nuclear norm minimization with application to image denoising[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2014:2862-2869
[7]Osher S,Burger M,Goldfarb D,et al.An iterative regularization method for total variation-based image restoration[J].Multiscale Modeling&Simulation,2005,4(2):460-489
[8]Rudin L I,Osher S,Fatemi E.Nonlinear total variation based noise removal algorithms[J].Physica D,1992,60(1-4):259-268
[9]Xie J Y,Xu L L,Chen E H.Image denoising and inpainting with deep neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems.Los Alamitos:IEEE Computer Society Press,2012,1:341-349
[10]Cui Z,Chang H,Shan S G,et al.Deep network cascade for image super-resolution[C]//Proceedings of European Conference on Computer Vision.Heidelberg:Springer,2014:49-64
[11]Xu J,Zhang L,Zuo W M,et al.Patch group based nonlocal self-similarity prior learning for image denoising[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2015:244-252
[12]Chen F,Zhang L,Yu H M.External patch prior guided internal clustering for image denoising[C]//Proceedings of the IEEEInternational Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2015:603-611
[13]Dong W S,Zhang L,Shi G M,et al.Nonlocally centralized sparse representation for image restoration[J].IEEE Transactions on Image Processing,2013,22(4):1620-1630
[14]Wang Z W,Liu D,Yang J C,et al.Deep networks for image super-resolution with sparse prior[C]//Proceedings of the IEEEInternational Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2015:370-378
[15]Dong C,Loy C C,He K M,et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307
[16]Huang Y,Wang W,Wang L.Bidirectional recurrent convolutional networks for multi-frame super-resolution[C]//Proceedings of Advances in Neural Information Processing Systems.Cambridge:MIT Press,2015:235-243
[17]Burger H C,Schuler C J,Harmeling S.Image denoising:can plain neural networks compete with BM3D?[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2012:2392-2399
[18]Jain V,Seung H S.Natural image denoising with convolutional networks[C]//Proceedings of Advances in Neural Information Processing System.Cambridge:MIT Press,2008:769-776
[19]Simonyan K,Zisserman A.Very deep convolutional networks for large-scale image recognition[C]//Proceedings of IEEEConference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2014:1-14
[20]Szegedy C,Liu W,Jia Y Q,et al.Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2015:1-9
[21]He K M,Zhang X Y,Ren S Q,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:770-778
[22]Howard A G,Zhu M L,Chen B,et al.MobileNets:efficient convolutional neural networks for mobile vision applications[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:1-9
[23]Iandola F N,Han S,Moskewicz M W,et al.Squeezenet:Alexnet-level accuracy with 50x fewer parameters and<0.5 mb model size[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2016:1-13
[24]Ren J,Chen X H,Liu J B,et al.Accurate single stage detector using recurrent rolling convolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:752-760
[2]Wang Z Y,Yang Y Z,Wang Z W,et al.Learning super-resolution jointly from external and internal examples[J].IEEETransactions on Image Processing,2015,24(11):4359-4371
[3]Gu S H,Zuo W M,Xie Q,et al.Convolutional sparse coding for image super-resolution[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2015:1823-1831
[4]Chatterjee P,Milanfar P.Clustering-based denoising with locally learned dictionaries[J].IEEE Transactions on Image Processing,2009,18(7):1438-1451
[5]Dabov K,Foi A,Katkovnik V,et al.Image denoising by sparse3-D transform-domain collaborative filtering[J].IEEE Transactions on Image Processing,2007,16(8):2080-2095
[6]Gu S H,Zhang L,Zuo W M,et al.Weighted nuclear norm minimization with application to image denoising[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2014:2862-2869
[7]Osher S,Burger M,Goldfarb D,et al.An iterative regularization method for total variation-based image restoration[J].Multiscale Modeling&Simulation,2005,4(2):460-489
[8]Rudin L I,Osher S,Fatemi E.Nonlinear total variation based noise removal algorithms[J].Physica D,1992,60(1-4):259-268
[9]Xie J Y,Xu L L,Chen E H.Image denoising and inpainting with deep neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems.Los Alamitos:IEEE Computer Society Press,2012,1:341-349
[10]Cui Z,Chang H,Shan S G,et al.Deep network cascade for image super-resolution[C]//Proceedings of European Conference on Computer Vision.Heidelberg:Springer,2014:49-64
[11]Xu J,Zhang L,Zuo W M,et al.Patch group based nonlocal self-similarity prior learning for image denoising[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2015:244-252
[12]Chen F,Zhang L,Yu H M.External patch prior guided internal clustering for image denoising[C]//Proceedings of the IEEEInternational Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2015:603-611
[13]Dong W S,Zhang L,Shi G M,et al.Nonlocally centralized sparse representation for image restoration[J].IEEE Transactions on Image Processing,2013,22(4):1620-1630
[14]Wang Z W,Liu D,Yang J C,et al.Deep networks for image super-resolution with sparse prior[C]//Proceedings of the IEEEInternational Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2015:370-378
[15]Dong C,Loy C C,He K M,et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307
[16]Huang Y,Wang W,Wang L.Bidirectional recurrent convolutional networks for multi-frame super-resolution[C]//Proceedings of Advances in Neural Information Processing Systems.Cambridge:MIT Press,2015:235-243
[17]Burger H C,Schuler C J,Harmeling S.Image denoising:can plain neural networks compete with BM3D?[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2012:2392-2399
[18]Jain V,Seung H S.Natural image denoising with convolutional networks[C]//Proceedings of Advances in Neural Information Processing System.Cambridge:MIT Press,2008:769-776
[19]Simonyan K,Zisserman A.Very deep convolutional networks for large-scale image recognition[C]//Proceedings of IEEEConference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2014:1-14
[20]Szegedy C,Liu W,Jia Y Q,et al.Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2015:1-9
[21]He K M,Zhang X Y,Ren S Q,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:770-778
[22]Howard A G,Zhu M L,Chen B,et al.MobileNets:efficient convolutional neural networks for mobile vision applications[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:1-9
[23]Iandola F N,Han S,Moskewicz M W,et al.Squeezenet:Alexnet-level accuracy with 50x fewer parameters and<0.5 mb model size[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2016:1-13
[24]Ren J,Chen X H,Liu J B,et al.Accurate single stage detector using recurrent rolling convolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:752-760