基于稀疏约束的图像超分辨率重建技术研究
|
摘要
图像超分辨率重建是利用同一场景中的一幅或一组低分辨率图像,结合一定的先验知识,重建一幅高分辨率图像的技术。该技术能够在不改变现有设备的条件下,有效提高图像的分辨率,具有广阔的应用前景。
本文首先简要介绍了图像超分辨率重建的现有算法及图像稀疏冗余模型的相关理论,然后针对图像非局部冗余稀疏和字典稀疏表示两个方面,进行超分辨率重建算法研究。主要工作包括3个方面：
1.提出改进的非局部迭代反投影(NLIBP:Non-Local Iterative Back-Projection)算法。通过自适应的控制参与NL后处理的像素点范围及优化相似度计算的方法,降低其计算复杂度,并抑制其部分过修正问题。实验结果表明,改进后的算法具有较快的重建速度,且具有较好的主客观重建质量。
2.简要介绍了基于稀疏约束的算法,并针对其计算复杂度高的问题,提出了一种基于K均值聚类的自适应快速重建方法。所提算法从两个方面降低其计算复杂度：(1)分类训练字典,对图像块归类重建,降低每个图像块所用字典的大小；(2)对图像块的特征进行分析,自适应的选择重建方法。实验结果表明,本文提出的快速重建方法在重建质量与原算法相当的前提下,可以在较大程度上降低重建时间。
3.针对基于稀疏字典约束的超分辨重建算法,提出了一种以非局部相似结构为导向的全局后处理方法,通过结合非局部均值(NLM:Non-Local Means)边缘去噪算法与改进的NLIBP算法,充分利用图像中的边缘信息、非局部相似结构,提高图像重建质量。实验结果表明,本文提出的后处理方法可以有效的改善图像边缘及整体上的平滑性,获得了更好的主客观重建质量。
Image super-resolution is to reconstruct a high resolution image from one or more low resolution images of the same scene by using some certain prior knowledge. This technology has a good prospect for its device independence and encouraging results.
This thesis first briefly introduces some classical algorithms and some theories about image sparse and redundant representation model, and then focuses on the research of image super-resolution based on image non-local redundancy sparsity and dictionary sparse representation. The three contributions presented in this thesis can be concluded as follows.
1. An improved non-local iterative back-projection algorithm (NLIBP) is proposed. It can reduce the computational complexity and avoid the over-correction for some interpolated pixels by adaptively controlling the number of pixels involved in the non-local modifying process and optimizing the similarity calculation between pixels. Experimental results show that this improved algorithm reduces the reconstruction complexity and generates images with higher subjective and objective quality.
2. An adaptive fast reconstruction method based on the K-Means clustering is presented to reduce the reconstruction computation complexity consumed by the algorithm based on the dictionary sparse constraint model. This presented algorithm reduces its complexity from two aspects. (1) Reduce the dictionary size for each image patch in the learning process by classifying the sampled raw patches in the dictionary training process. (2) Adaptively select the reconstruction algorithm according to the features existed in each patch. Experimental results show that this proposed fast reconstruction method takes much less time while generating images equivalent to the original algorithm.
3. A global post-processing method guided by non-local similarity structure is proposed to improve the reconstruction quality gained by the algorithm based on dictionary sparse constraint model. It makes full use of the edges and the non-local similarity structure existed in the low resolution image and combines the non-local means (NLM) denoising algorithm and the improved NLIBP algorithm successfully. Experimental results show that this proposed post-processing method can effectively improve image edges and its overall smoothness, generating images with higher subjective and objective quality.
本文首先简要介绍了图像超分辨率重建的现有算法及图像稀疏冗余模型的相关理论,然后针对图像非局部冗余稀疏和字典稀疏表示两个方面,进行超分辨率重建算法研究。主要工作包括3个方面：
1.提出改进的非局部迭代反投影(NLIBP:Non-Local Iterative Back-Projection)算法。通过自适应的控制参与NL后处理的像素点范围及优化相似度计算的方法,降低其计算复杂度,并抑制其部分过修正问题。实验结果表明,改进后的算法具有较快的重建速度,且具有较好的主客观重建质量。
2.简要介绍了基于稀疏约束的算法,并针对其计算复杂度高的问题,提出了一种基于K均值聚类的自适应快速重建方法。所提算法从两个方面降低其计算复杂度：(1)分类训练字典,对图像块归类重建,降低每个图像块所用字典的大小；(2)对图像块的特征进行分析,自适应的选择重建方法。实验结果表明,本文提出的快速重建方法在重建质量与原算法相当的前提下,可以在较大程度上降低重建时间。
3.针对基于稀疏字典约束的超分辨重建算法,提出了一种以非局部相似结构为导向的全局后处理方法,通过结合非局部均值(NLM:Non-Local Means)边缘去噪算法与改进的NLIBP算法,充分利用图像中的边缘信息、非局部相似结构,提高图像重建质量。实验结果表明,本文提出的后处理方法可以有效的改善图像边缘及整体上的平滑性,获得了更好的主客观重建质量。
Image super-resolution is to reconstruct a high resolution image from one or more low resolution images of the same scene by using some certain prior knowledge. This technology has a good prospect for its device independence and encouraging results.
This thesis first briefly introduces some classical algorithms and some theories about image sparse and redundant representation model, and then focuses on the research of image super-resolution based on image non-local redundancy sparsity and dictionary sparse representation. The three contributions presented in this thesis can be concluded as follows.
1. An improved non-local iterative back-projection algorithm (NLIBP) is proposed. It can reduce the computational complexity and avoid the over-correction for some interpolated pixels by adaptively controlling the number of pixels involved in the non-local modifying process and optimizing the similarity calculation between pixels. Experimental results show that this improved algorithm reduces the reconstruction complexity and generates images with higher subjective and objective quality.
2. An adaptive fast reconstruction method based on the K-Means clustering is presented to reduce the reconstruction computation complexity consumed by the algorithm based on the dictionary sparse constraint model. This presented algorithm reduces its complexity from two aspects. (1) Reduce the dictionary size for each image patch in the learning process by classifying the sampled raw patches in the dictionary training process. (2) Adaptively select the reconstruction algorithm according to the features existed in each patch. Experimental results show that this proposed fast reconstruction method takes much less time while generating images equivalent to the original algorithm.
3. A global post-processing method guided by non-local similarity structure is proposed to improve the reconstruction quality gained by the algorithm based on dictionary sparse constraint model. It makes full use of the edges and the non-local similarity structure existed in the low resolution image and combines the non-local means (NLM) denoising algorithm and the improved NLIBP algorithm successfully. Experimental results show that this proposed post-processing method can effectively improve image edges and its overall smoothness, generating images with higher subjective and objective quality.
引文
[1]R. C. Gonzalez, R. E. Woods. Digital image processing[M]. Upper Saddle River, N.J:Prentice Hall,2002.
[2]S. C. Park, M. K. Park, M. G. Kang. Super-resolution image reconstruction:a technical review[J], IEEE Signal Processing Magazine,5(21):21-36,2003.
[3]R. Y. Tsai, T. S. Huang. Multiple frame image restoration and registration[C]. In Advances in Computer Vision and Image Processing, pp.317-339,1984.
[4]W. E. Vinje, J. L. Gallant. sparse coding and decorrelation in primary visual cortex during natural vision[J]. Science,287(5456):1273-1276,2000.
[5]S. Nirenberg. S. M. Carcieri, A. L. Jacobs, P. E. Latham. Retinal ganglion cells act largely as independent encoders[J]. Nature,411(6838):698-701,2001.
[6]B. A. Olshausen, D. J. Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images[J]. Nature,381(6583):607-609,1996.
[7]R. Rubinstein, A. M. Bruckstein and M. Elad. Dictionaries for sparse representation modeling[J]. IEEE Proceedings,98(6):1045-1057, June 2010.
[8]W. Dong, L. Zhang, G. Shi, and X. Wu. Image deblurring and supper-resolution by adaptive sparse domain selection and adaptive regularization[J]. IEEE Transactions on Image Processing,20(7):1838-1857, July 2011.
[9]W. Dong, G. Shi, L. Zhang, and X. Wu. Super-resolution with nonlocal regularized sparse representation[J]. in SPIE Visual Communications and Image Processing, July 2010.
[10]J. Yang, J. Wright, T. Huang, and Y. Ma. Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing,19(11):2861-2873, November 2010.
[11]L. J. Harris. Diffraction and resolving power[J]. J. Opt. Soc. Am.54(7):931-933,1964.
[12]W. J. Goodman. Introduction to fourier optics[M]. Roberts and Company Publishers,1968.
[13]R. G. Keys. Cubic convolution interpolation for digital image processing[J]. IEEE Transactions on Acoustics, Speech and Signal Processing,29(6):1153-1160, December 1981.
[14]H. S. Hou and H. C. Andrews. Cubic splines for image interpolation and digital filtering[J]. IEEE Transactions on Acoustics, Speech and Signal Processing,26(6):508-517, December 1987.
[15]X. Li and M. T. Orchard. New edge-directed interpolation[J]. IEEE Transactions on Image Processing, 10(10):1521-1527, October 2001.
[16]X. Zhang and X. Wu. Image interpolation by adaptive 2-D autoregressive modeling and soft-decision estimation[J]. IEEE Transactions on Image Processing,17(6):887-896, June 2008.
[17]H. Stark, P. Oskoui. High-resolution image recovery from image-planear rays, using convex projections[J]. Journal of the Opt.Soc.of America,6(11):1715-1726,1989.
[18]J. A. Patti, I. M. Sezan, M. A. Tekalp. Super-resolution video reconstruction with arbitrary sampling lattices and nonzero aperture time[J]. IEEE Transactions on Image Processing,6(8):1064-1076,1977.
[19]M. Irani and S. Peleg. Improving resolution by image registration[J]. CVGIP:Graphical Models and Image Processing,53(3):231-239,1991.
[20]R. Schultz, R. Stevenson. Bayesian estimation of sub-pixel resolution motion fiesld and high-resolution video stills[C]. IEEE Int. Conf. on Image Processing, vol.3, pp.62-65.1997.
[21]M. Elad, A. Feuer. Restoration of a single super-resolution image from several blurred, noisy and undersampled measured images[J]. IEEE Transactions on Image Processing,6(12):1646-1658,1997.
[22]W. Dong, L. Zhang, G. Shi, and X. Wu. Nonlocal back-projection for adaptive image enlargement[C]. IEEE International Conference on Image Processing, pp.349-352, November 2009.
[23]W. T. Freeman, T. R. Johns, and E. C. Pasztor. Example-based super-resolution[J]. IEEE Computer Graphics and Applications,22(2):56-65, August 2002.
[24]H. Chang, D.-Y. Yeung, and Y. Xiong. Super-resolution through neighbor embedding[C]. IEEE Computer Society Conference on CVPR,2004.
[25]J. Yang, J. Wright, T. Huang, and Y. Ma. Image super-resolution as sparse representation of raw image patches[C]. IEEE Conference on CVPR, pp.1-8, June 2003.
[26]朱秀昌,刘峰,胡栋.数字图像处理与图像通信[M].北京：北京邮电大学出版社,2002.
[27]I. Avcibas, B. Sankur, K. Sayood. Statistical evaluation of image quality measures[J]. Journal of Electronic Imaging,11(2):206-213,2002.
[28]Lin Zhang, Lei Zhang, X. Mou and D. Zhang. FSIM:A feature similarity index for image quality assessment J]. IEEE Transactions on Image Processing,2011.
[29]宋慧慧.基于稀疏表示的图像超分辨率重建算法研究[D].中国科学技术大学硕士学位论文,2011.
[30]G. Davis. Adaptive nonlinear approximations[D].New York:New York University,1994.
[31]B. K. Natarajan. Sparse approximate solutions to linear systems[J]. SIAM Journal on Computing, 24(2):227-234,1995.
[32]J. A. Tropp. Greed is good:Algorithmic results for sparse approximation[J]. IEEE Transactions on Information Theory,50(10):2231-2242, October.2004.
[33]S. Mallat and Z. Zhang. Matching pursuits with time-frequency dictionaries[J]. IEEE Transactions on Signal Processing,41(12):3397-3415, December 1993.
[34]Y. C. Pati, R. Rezaiifar, and P. S. Krishnaprasad. Orthogonal matching pursuit:Recursive function approximation with applications to wavelet decomposition[C]. in Conf. Rec.27th Asilomar Conf. Signals, Syst. Comput., vol.1, pp.30-44, November 1993.
[35]S. Chen, S. A. Billings, and W. Luo. Orthogonal least squares methods and their application to non-linear system identification[J], International Journal of Control,50(5):1873-96,1989.
[36]D. L. Donoho, Y. Tsaig, I. Drori. J. L. Starck. Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit[R]. Tech. Rep.2006-2, Department of Statisties, Stanford University, 2006.
[37]M. Aharon, M. Elad and A. Bruckstein. K-SVD:an algorithm for designing overcomplete dictionaries for sparse representation[J]. IEEE Transactions on Signal Processing,54(11), November 2006.
[38]K. Engan, S. O. Aase, and J. H. Husoy. Method of optimal directions for frame design[C]. IEEE International Conference on Acoustics, Speech, and Signal Processing, vol.5, pp.2443-2446,1999.
[39]K. Engan, B. D. Rao, and K. Kreutz-Delgado. Frame design using FOCUSS with method of optimal directions (MOD)[J]. Proc. Norwegian Signal Processing Symposium, pp.65-69,1999.
[40]J. Mairal, G. Sapiro, and M. Elad. Learning multiscale sparse representations for image and video restoration. SI AM Multiscale Modeling and Simulation,7(1):214-241,2008.
[41]R. Rubinstein, M. Zibulevsky, and M. Elad. Learning sparse dictionaries for sparse signal representation[J]. IEEE Transactions on Signal Processing. To appear.
[42]H. Lee, A. Battle, R. Raina, Andrew Y. Ng. Efficient sparse coding algorithms [J]. in NIPS,2007.
[43]D. L. Donoho. Compressed sensing[J]. IEEE Transactions.on Information Theory,52(4):1289-1306,2006.
[44]陈少冲.一种自适应学习的图像超分辨率重建算法研究[D].西安电子科技大学硕士学位论文,2011.
[45]A. Gersho and R. M. Gray. Vector quantization and signal compression[M]. Norwell, MA:Kluwer Academic, 1991.
[46]A. Buades, B. Coll and J.-M. Morel. A non-local algorithm for image denoising[C]. IEEE Computer Society Conference on CVPR, pp.60-65, June 2005.
[2]S. C. Park, M. K. Park, M. G. Kang. Super-resolution image reconstruction:a technical review[J], IEEE Signal Processing Magazine,5(21):21-36,2003.
[3]R. Y. Tsai, T. S. Huang. Multiple frame image restoration and registration[C]. In Advances in Computer Vision and Image Processing, pp.317-339,1984.
[4]W. E. Vinje, J. L. Gallant. sparse coding and decorrelation in primary visual cortex during natural vision[J]. Science,287(5456):1273-1276,2000.
[5]S. Nirenberg. S. M. Carcieri, A. L. Jacobs, P. E. Latham. Retinal ganglion cells act largely as independent encoders[J]. Nature,411(6838):698-701,2001.
[6]B. A. Olshausen, D. J. Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images[J]. Nature,381(6583):607-609,1996.
[7]R. Rubinstein, A. M. Bruckstein and M. Elad. Dictionaries for sparse representation modeling[J]. IEEE Proceedings,98(6):1045-1057, June 2010.
[8]W. Dong, L. Zhang, G. Shi, and X. Wu. Image deblurring and supper-resolution by adaptive sparse domain selection and adaptive regularization[J]. IEEE Transactions on Image Processing,20(7):1838-1857, July 2011.
[9]W. Dong, G. Shi, L. Zhang, and X. Wu. Super-resolution with nonlocal regularized sparse representation[J]. in SPIE Visual Communications and Image Processing, July 2010.
[10]J. Yang, J. Wright, T. Huang, and Y. Ma. Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing,19(11):2861-2873, November 2010.
[11]L. J. Harris. Diffraction and resolving power[J]. J. Opt. Soc. Am.54(7):931-933,1964.
[12]W. J. Goodman. Introduction to fourier optics[M]. Roberts and Company Publishers,1968.
[13]R. G. Keys. Cubic convolution interpolation for digital image processing[J]. IEEE Transactions on Acoustics, Speech and Signal Processing,29(6):1153-1160, December 1981.
[14]H. S. Hou and H. C. Andrews. Cubic splines for image interpolation and digital filtering[J]. IEEE Transactions on Acoustics, Speech and Signal Processing,26(6):508-517, December 1987.
[15]X. Li and M. T. Orchard. New edge-directed interpolation[J]. IEEE Transactions on Image Processing, 10(10):1521-1527, October 2001.
[16]X. Zhang and X. Wu. Image interpolation by adaptive 2-D autoregressive modeling and soft-decision estimation[J]. IEEE Transactions on Image Processing,17(6):887-896, June 2008.
[17]H. Stark, P. Oskoui. High-resolution image recovery from image-planear rays, using convex projections[J]. Journal of the Opt.Soc.of America,6(11):1715-1726,1989.
[18]J. A. Patti, I. M. Sezan, M. A. Tekalp. Super-resolution video reconstruction with arbitrary sampling lattices and nonzero aperture time[J]. IEEE Transactions on Image Processing,6(8):1064-1076,1977.
[19]M. Irani and S. Peleg. Improving resolution by image registration[J]. CVGIP:Graphical Models and Image Processing,53(3):231-239,1991.
[20]R. Schultz, R. Stevenson. Bayesian estimation of sub-pixel resolution motion fiesld and high-resolution video stills[C]. IEEE Int. Conf. on Image Processing, vol.3, pp.62-65.1997.
[21]M. Elad, A. Feuer. Restoration of a single super-resolution image from several blurred, noisy and undersampled measured images[J]. IEEE Transactions on Image Processing,6(12):1646-1658,1997.
[22]W. Dong, L. Zhang, G. Shi, and X. Wu. Nonlocal back-projection for adaptive image enlargement[C]. IEEE International Conference on Image Processing, pp.349-352, November 2009.
[23]W. T. Freeman, T. R. Johns, and E. C. Pasztor. Example-based super-resolution[J]. IEEE Computer Graphics and Applications,22(2):56-65, August 2002.
[24]H. Chang, D.-Y. Yeung, and Y. Xiong. Super-resolution through neighbor embedding[C]. IEEE Computer Society Conference on CVPR,2004.
[25]J. Yang, J. Wright, T. Huang, and Y. Ma. Image super-resolution as sparse representation of raw image patches[C]. IEEE Conference on CVPR, pp.1-8, June 2003.
[26]朱秀昌,刘峰,胡栋.数字图像处理与图像通信[M].北京：北京邮电大学出版社,2002.
[27]I. Avcibas, B. Sankur, K. Sayood. Statistical evaluation of image quality measures[J]. Journal of Electronic Imaging,11(2):206-213,2002.
[28]Lin Zhang, Lei Zhang, X. Mou and D. Zhang. FSIM:A feature similarity index for image quality assessment J]. IEEE Transactions on Image Processing,2011.
[29]宋慧慧.基于稀疏表示的图像超分辨率重建算法研究[D].中国科学技术大学硕士学位论文,2011.
[30]G. Davis. Adaptive nonlinear approximations[D].New York:New York University,1994.
[31]B. K. Natarajan. Sparse approximate solutions to linear systems[J]. SIAM Journal on Computing, 24(2):227-234,1995.
[32]J. A. Tropp. Greed is good:Algorithmic results for sparse approximation[J]. IEEE Transactions on Information Theory,50(10):2231-2242, October.2004.
[33]S. Mallat and Z. Zhang. Matching pursuits with time-frequency dictionaries[J]. IEEE Transactions on Signal Processing,41(12):3397-3415, December 1993.
[34]Y. C. Pati, R. Rezaiifar, and P. S. Krishnaprasad. Orthogonal matching pursuit:Recursive function approximation with applications to wavelet decomposition[C]. in Conf. Rec.27th Asilomar Conf. Signals, Syst. Comput., vol.1, pp.30-44, November 1993.
[35]S. Chen, S. A. Billings, and W. Luo. Orthogonal least squares methods and their application to non-linear system identification[J], International Journal of Control,50(5):1873-96,1989.
[36]D. L. Donoho, Y. Tsaig, I. Drori. J. L. Starck. Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit[R]. Tech. Rep.2006-2, Department of Statisties, Stanford University, 2006.
[37]M. Aharon, M. Elad and A. Bruckstein. K-SVD:an algorithm for designing overcomplete dictionaries for sparse representation[J]. IEEE Transactions on Signal Processing,54(11), November 2006.
[38]K. Engan, S. O. Aase, and J. H. Husoy. Method of optimal directions for frame design[C]. IEEE International Conference on Acoustics, Speech, and Signal Processing, vol.5, pp.2443-2446,1999.
[39]K. Engan, B. D. Rao, and K. Kreutz-Delgado. Frame design using FOCUSS with method of optimal directions (MOD)[J]. Proc. Norwegian Signal Processing Symposium, pp.65-69,1999.
[40]J. Mairal, G. Sapiro, and M. Elad. Learning multiscale sparse representations for image and video restoration. SI AM Multiscale Modeling and Simulation,7(1):214-241,2008.
[41]R. Rubinstein, M. Zibulevsky, and M. Elad. Learning sparse dictionaries for sparse signal representation[J]. IEEE Transactions on Signal Processing. To appear.
[42]H. Lee, A. Battle, R. Raina, Andrew Y. Ng. Efficient sparse coding algorithms [J]. in NIPS,2007.
[43]D. L. Donoho. Compressed sensing[J]. IEEE Transactions.on Information Theory,52(4):1289-1306,2006.
[44]陈少冲.一种自适应学习的图像超分辨率重建算法研究[D].西安电子科技大学硕士学位论文,2011.
[45]A. Gersho and R. M. Gray. Vector quantization and signal compression[M]. Norwell, MA:Kluwer Academic, 1991.
[46]A. Buades, B. Coll and J.-M. Morel. A non-local algorithm for image denoising[C]. IEEE Computer Society Conference on CVPR, pp.60-65, June 2005.