基于结构自相似性和形变块特征的单幅图像超分辨率算法
|
摘要
针对单幅图像超分辨率(SR)复原样本资源不足和抗噪性差的问题,提出一种基于结构自相似和形变块特征的单幅图像超分辨率算法。首先,该方法通过构建尺度模型,尽可能地扩展搜索空间,克服单幅图像超分辨率训练样本不足的缺陷;接着,通过样例块的几何形变提升了局限性的内部字典大小;最后,为了提升重建图片的抗噪性,利用组稀疏学习字典来重建图像。实验结果表明:与Bicubic、稀疏字典学习(Sc SR)算法和基于卷积神经网络的超分辨率(SRCNN)等优秀字典学习算法相比,所提算法可以得到主观视觉效果更为清晰和客观评价更高的超分辨率图像,峰值信噪比(PSNR)平均约提升了0. 35 d B。另外所提算法通过几何形变的方式扩展了字典规模和搜索的准确性,在算法时间消耗上平均约减少了80 s。
To solve the problem of insufficient sample resources and poor noise immunity for single image Super Resolution( SR) restoration, a single image super-resolution algorithm based on structural self-similarity and deformation block feature was proposed. Firstly, a scale model was constructed to expand search space as much as possible and overcome the shortcomings of lack of a single image super-resolution training sample. Secondly, the limited internal dictionary size was increased by geometric deformation of sample block. Finally, in order to improve anti-noise performance of reconstructed picture, the group sparse learning dictionary was used to reconstruct image. The experimental results show that compared with the excellent algorithms such as Bicubic, Sparse coding Super Resolution( Sc SR) algorithm and Super-Resolution Convolutional Neural Network( SRCNN) algorithm, the super-resolution images with more subjective visual effects and higher objective evaluation can be obtained, the Peak Signal-To-Noise Ratio( PSNR) of the proposed algorithm is increased by about0. 35 d B on average. In addition, the scale of dictionary is expanded and the accuracy of search is increased by means of geometric deformation, and the time consumption of algorithm is averagely reduced by about 80 s.
To solve the problem of insufficient sample resources and poor noise immunity for single image Super Resolution( SR) restoration, a single image super-resolution algorithm based on structural self-similarity and deformation block feature was proposed. Firstly, a scale model was constructed to expand search space as much as possible and overcome the shortcomings of lack of a single image super-resolution training sample. Secondly, the limited internal dictionary size was increased by geometric deformation of sample block. Finally, in order to improve anti-noise performance of reconstructed picture, the group sparse learning dictionary was used to reconstruct image. The experimental results show that compared with the excellent algorithms such as Bicubic, Sparse coding Super Resolution( Sc SR) algorithm and Super-Resolution Convolutional Neural Network( SRCNN) algorithm, the super-resolution images with more subjective visual effects and higher objective evaluation can be obtained, the Peak Signal-To-Noise Ratio( PSNR) of the proposed algorithm is increased by about0. 35 d B on average. In addition, the scale of dictionary is expanded and the accuracy of search is increased by means of geometric deformation, and the time consumption of algorithm is averagely reduced by about 80 s.
引文
[1]FREEMAN W T, JONES T R, PASZTOR E C. Example based super resolution[J]. IEEE Computer Graphics and Applications,2002, 22(2):56-65.
[2]CHANG H, YEUNG D Y, XIONG Y. Super-resolution through neighbor embedding[C]//Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Washington, DC:IEEE Computer Society, 2004:I-275-I-282.
[3]YANG J, WRIGHT J, HUANG T, et al. Image super-resolution via spare representation[J]. IEEE Transactions on Image Processing,2010, 19(11):2861-2873.
[4]YANG C Y, YANG M H. Fast direct super-resolution by simple functions[C]//Proceedings of the 2013 IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 2013:561-568.
[5]TIMOFTE R, SMET V, GOOL L. Anchored neighborhood regression for fast example-based super-resolution[C]//Proceedings of the2013 IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 2013:1920-1927.
[6]TIMOFTE R, SMET V, GOOL L. A+:adjusted anchored neighborhood regression for fast super resolution[C]//Proceedings of the2014 Asian Conference on Computer Vision. Berlin:Spring, 2014:111-126.
[7]DONG C, LOY C C, HE K, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
[8]MEHRAN E, VRSCAY E R. Solving the inverse problem of image zooming usingSelf-Examples[M]//Image Analysis and Recognition. Berlin:Springer, 2007:117-130.
[9]GKASNER D, BAGON S, IRANI M. Super-resolution from a single image[C]//Proceedings of the 2009 International Conference on Computer Vision. Piscataway, NJ:IEEE, 2009:349-356.
[10]FREEDMAN G, FATTAL R. Image and video upscaling from local self-examples[J]. ACM Transactions on Graphics, 2011,30(2):1-11.
[11]SINGH A, AHUJA N. Super-resolution using sub-band self-similarity[C]//ACCV 2014:Proceedings of the 2014 Asian Conference on Computer Vision. Berlin:Springer, 2014:552-568.
[12]ZONTAK M, IRANI M. Internal statistics of a single natural image[C]//CVPR'11:Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2011:977-984.
[13]向文,张灵,陈云华,等.结合结构自相似性和卷积网络的单幅图像超分辨率[J].计算机应用, 2018, 38(3):854-858.(XIANG W, ZHANG L, CHEN Y H, et al. Combined with structural self-similarity and convolution of a single image super resolution[J]. Journal of Computer Applications, 2018, 38(3):854-858.)
[14]肖进胜,刘恩雨,朱力,等.改进的基于卷积神经网络的图像超分辨率算法[J].光学学报,2017,37(3):103-111.(XIAO J S,LIU E Y, ZHU L, et al. Improved image super-resolution algorithm based on convolutional neural network[J]. Acta Optica Sinica,2017,37(3):103-111.)
[15]HUANG J B, KANG S B, AHUJA N, et al. Image completion using planar structure guidance[J]. ACM Transactions on Graphics, 2014, 33(4):1-10.
[16]FREEMAN W T, JONES T R, PASZTOR E C. Example-based super-resolution[J]. IEEE Computer Graphics&Applications,2002, 22(2):56-65
[17]YANG J, WRIGHT J, HUANG T, et al. Image super-resolution as sparse representation of raw image patches[C]//CVPR 2008:Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society,2008:1-8.
[18]AHARON M, ELAD M, BRUCKSTEIN A. K-SVD:an algorithm for designing overcomplete dictionaries for sparse representation[J]. IEEE Transactions on Signal Processing, 2006, 54(11):4311-4322.
[19]DARABI S, SHECHTMAN E, BARNES C, et al. Image melding:combining inconsistent images using patch-based synthesis[J].ACM Transactions on Graphics, 2012, 31(4):1-10.
[20]HACOHEN Y, FATTAL R, LISCHINSKI D. Image upsampling via texture hallucination[C]//Proceedings of the 2010 IEEE International Conference on Computational Photography. Piscataway,NJ:IEEE, 2010:1-8.
[21]BARNES C, SHECHTMAN E, FINKELSTEIN A, et al. PatchMatch:a randomized correspondence algorithm for structural image editing[J]. ACM Transactions on Graphics, 2009, 28(3):1-11.
[22]YANG J, LIN Z, COHEN S. Fast image super-resolution based on in-place example regression[C]//Proceedings of the 2013 Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2013:1059-1066.
[23]KIM K I, KWON Y. Single-image super-resolution using sparse regression and natural image prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010,32(6):1127-1133.
[2]CHANG H, YEUNG D Y, XIONG Y. Super-resolution through neighbor embedding[C]//Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Washington, DC:IEEE Computer Society, 2004:I-275-I-282.
[3]YANG J, WRIGHT J, HUANG T, et al. Image super-resolution via spare representation[J]. IEEE Transactions on Image Processing,2010, 19(11):2861-2873.
[4]YANG C Y, YANG M H. Fast direct super-resolution by simple functions[C]//Proceedings of the 2013 IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 2013:561-568.
[5]TIMOFTE R, SMET V, GOOL L. Anchored neighborhood regression for fast example-based super-resolution[C]//Proceedings of the2013 IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 2013:1920-1927.
[6]TIMOFTE R, SMET V, GOOL L. A+:adjusted anchored neighborhood regression for fast super resolution[C]//Proceedings of the2014 Asian Conference on Computer Vision. Berlin:Spring, 2014:111-126.
[7]DONG C, LOY C C, HE K, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
[8]MEHRAN E, VRSCAY E R. Solving the inverse problem of image zooming usingSelf-Examples[M]//Image Analysis and Recognition. Berlin:Springer, 2007:117-130.
[9]GKASNER D, BAGON S, IRANI M. Super-resolution from a single image[C]//Proceedings of the 2009 International Conference on Computer Vision. Piscataway, NJ:IEEE, 2009:349-356.
[10]FREEDMAN G, FATTAL R. Image and video upscaling from local self-examples[J]. ACM Transactions on Graphics, 2011,30(2):1-11.
[11]SINGH A, AHUJA N. Super-resolution using sub-band self-similarity[C]//ACCV 2014:Proceedings of the 2014 Asian Conference on Computer Vision. Berlin:Springer, 2014:552-568.
[12]ZONTAK M, IRANI M. Internal statistics of a single natural image[C]//CVPR'11:Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2011:977-984.
[13]向文,张灵,陈云华,等.结合结构自相似性和卷积网络的单幅图像超分辨率[J].计算机应用, 2018, 38(3):854-858.(XIANG W, ZHANG L, CHEN Y H, et al. Combined with structural self-similarity and convolution of a single image super resolution[J]. Journal of Computer Applications, 2018, 38(3):854-858.)
[14]肖进胜,刘恩雨,朱力,等.改进的基于卷积神经网络的图像超分辨率算法[J].光学学报,2017,37(3):103-111.(XIAO J S,LIU E Y, ZHU L, et al. Improved image super-resolution algorithm based on convolutional neural network[J]. Acta Optica Sinica,2017,37(3):103-111.)
[15]HUANG J B, KANG S B, AHUJA N, et al. Image completion using planar structure guidance[J]. ACM Transactions on Graphics, 2014, 33(4):1-10.
[16]FREEMAN W T, JONES T R, PASZTOR E C. Example-based super-resolution[J]. IEEE Computer Graphics&Applications,2002, 22(2):56-65
[17]YANG J, WRIGHT J, HUANG T, et al. Image super-resolution as sparse representation of raw image patches[C]//CVPR 2008:Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society,2008:1-8.
[18]AHARON M, ELAD M, BRUCKSTEIN A. K-SVD:an algorithm for designing overcomplete dictionaries for sparse representation[J]. IEEE Transactions on Signal Processing, 2006, 54(11):4311-4322.
[19]DARABI S, SHECHTMAN E, BARNES C, et al. Image melding:combining inconsistent images using patch-based synthesis[J].ACM Transactions on Graphics, 2012, 31(4):1-10.
[20]HACOHEN Y, FATTAL R, LISCHINSKI D. Image upsampling via texture hallucination[C]//Proceedings of the 2010 IEEE International Conference on Computational Photography. Piscataway,NJ:IEEE, 2010:1-8.
[21]BARNES C, SHECHTMAN E, FINKELSTEIN A, et al. PatchMatch:a randomized correspondence algorithm for structural image editing[J]. ACM Transactions on Graphics, 2009, 28(3):1-11.
[22]YANG J, LIN Z, COHEN S. Fast image super-resolution based on in-place example regression[C]//Proceedings of the 2013 Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2013:1059-1066.
[23]KIM K I, KWON Y. Single-image super-resolution using sparse regression and natural image prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010,32(6):1127-1133.