超分辨率图像重建算法研究
|
摘要
图像,作为人类认识世界最主要的工具,其所含的信息内容是非常丰富的,在信息传递的过程中扮演着十分重要的角色。图像的分辨率是衡量图像包含信息多少的重要指标,图像的分辨率越高,代表图像包含的信息也就越多,图像反应的细节也就越丰富,因此在图像的应用领域中,人们都期望获得分辨率较高的图像。然而,由于成像系统本身条件以及外界因素的限制,使得获得的图像的分辨率降低,影响图像在实际中的广泛应用。
从硬件方面来说,可以通过减小像素尺寸,增大芯片尺寸或改变探测元排列方式这三种方法来实现对图像分辨率的改善。但是这些方法都受到传感器制造技术和硬件发展水平的限制,很难更进一步提高图像的分辨率。因此,出现了一种利用信号处理技术,从多帧观测到的低分辨率图像得到高分辨率图像的方法,即超分辨率图像重建技术。
本文首先对超分辨率图像重建的含义,研究状况及其应用方向进行了简单概述,随后利用数学方法对超分辨率图像重建问题进行了描述和数学建模,并在此基础上对常用的频率域算法和空间域算法进行了分类描述,给出了重建的一般步骤。另外,由于低分辨率图像会受到多种因素的影响而使得有用信息变得比较模糊,所以必须在超分辨率图像重建之前,对低分辨率图像进行预处理(增强,去噪,不良帧的剔除,图像配准等),为后续处理打好基础。其次,在已有运动估计算法的基础上,将块匹配算法与图像多分辨率分解相结合,有效地降低了原有运动估计算法的运算量。并在此基础上提出了一种基于S+P变换和插值处理(文中涉及的插值算法有：最近邻插值、双线型插值和Bezier曲面插值)的超分辨率图像重建算法。最后,通过对本文算法进行实验仿真,结果证明,无论是从主观还是客观方面来说,本文算法都取得了不错的效果。然后通过对整篇文章的系统总结,对超分辨率重建未来的发展方向进行了展望。
As the most important learning tool of human, image contains much information and plays a very important role in information transmission. As known, image resolution is an important index which measures how much information the image contains. Higher resolution means more information the image contains, that is to say the detail of the image is much richer. So, everyone wants to obtain the image with high resolution in the region of image processing. However, because of the limits from imaging system and the external factors, the images with low resolution are obtained, which affect the wild use of image.
From the view of hardware, we can improve the image resolution by reducing the pixel size, increasing the chip size and changing the arrangement of detection element. Due to the limits of sensor manufacturing technology and hardware development level, it is very hard to further improve the image resolution. Therefore, someone proposed a new method named super resolution image reconstruction technology which would obtain the image with high resolution from the multiple low resolution images using the signal processing technology.
In this paper, the concept, research condition and the application of super resolution image reconstruction were introduced simply, and the formation process of image was modeled using the math method firstly. And then, the general steps of reconstruction and the commonly used frequency domain algorithm and special domain algorithm were briefly described. Besides, due to the influence of many factors, low resolution images were blurred and the information of the images was limited. So, it was necessary to do preprocessing before super resolution reconstruction which included enhancement, de-noising, bad frame elimination and image registration, etc. It laid a good foundation for the following processing. Then, according to the existing methods, multi-resolution images registration method based on the block matching was proposed, and then the new super-resolution reconstruction algorithm based on the S + P transformation and interpolation was presented. The nearest neighbor interpolation, bilinear interpolation and Bezier surface interpolation were referred in the paper. Finally, the simulation experiments of the proposed algorithm were done and the results were analyzed using the subjective and objective evaluation indexes. It was proved that the proposed algorithm did well in the experiment. At last, based on the summary of the paper, the future development direction of super-resolution image reconstruction was discussed.
从硬件方面来说,可以通过减小像素尺寸,增大芯片尺寸或改变探测元排列方式这三种方法来实现对图像分辨率的改善。但是这些方法都受到传感器制造技术和硬件发展水平的限制,很难更进一步提高图像的分辨率。因此,出现了一种利用信号处理技术,从多帧观测到的低分辨率图像得到高分辨率图像的方法,即超分辨率图像重建技术。
本文首先对超分辨率图像重建的含义,研究状况及其应用方向进行了简单概述,随后利用数学方法对超分辨率图像重建问题进行了描述和数学建模,并在此基础上对常用的频率域算法和空间域算法进行了分类描述,给出了重建的一般步骤。另外,由于低分辨率图像会受到多种因素的影响而使得有用信息变得比较模糊,所以必须在超分辨率图像重建之前,对低分辨率图像进行预处理(增强,去噪,不良帧的剔除,图像配准等),为后续处理打好基础。其次,在已有运动估计算法的基础上,将块匹配算法与图像多分辨率分解相结合,有效地降低了原有运动估计算法的运算量。并在此基础上提出了一种基于S+P变换和插值处理(文中涉及的插值算法有：最近邻插值、双线型插值和Bezier曲面插值)的超分辨率图像重建算法。最后,通过对本文算法进行实验仿真,结果证明,无论是从主观还是客观方面来说,本文算法都取得了不错的效果。然后通过对整篇文章的系统总结,对超分辨率重建未来的发展方向进行了展望。
As the most important learning tool of human, image contains much information and plays a very important role in information transmission. As known, image resolution is an important index which measures how much information the image contains. Higher resolution means more information the image contains, that is to say the detail of the image is much richer. So, everyone wants to obtain the image with high resolution in the region of image processing. However, because of the limits from imaging system and the external factors, the images with low resolution are obtained, which affect the wild use of image.
From the view of hardware, we can improve the image resolution by reducing the pixel size, increasing the chip size and changing the arrangement of detection element. Due to the limits of sensor manufacturing technology and hardware development level, it is very hard to further improve the image resolution. Therefore, someone proposed a new method named super resolution image reconstruction technology which would obtain the image with high resolution from the multiple low resolution images using the signal processing technology.
In this paper, the concept, research condition and the application of super resolution image reconstruction were introduced simply, and the formation process of image was modeled using the math method firstly. And then, the general steps of reconstruction and the commonly used frequency domain algorithm and special domain algorithm were briefly described. Besides, due to the influence of many factors, low resolution images were blurred and the information of the images was limited. So, it was necessary to do preprocessing before super resolution reconstruction which included enhancement, de-noising, bad frame elimination and image registration, etc. It laid a good foundation for the following processing. Then, according to the existing methods, multi-resolution images registration method based on the block matching was proposed, and then the new super-resolution reconstruction algorithm based on the S + P transformation and interpolation was presented. The nearest neighbor interpolation, bilinear interpolation and Bezier surface interpolation were referred in the paper. Finally, the simulation experiments of the proposed algorithm were done and the results were analyzed using the subjective and objective evaluation indexes. It was proved that the proposed algorithm did well in the experiment. At last, based on the summary of the paper, the future development direction of super-resolution image reconstruction was discussed.
引文
[1]贺兴华,周媛媛,王继阳等.Matlab7.x图像处理.北京,人民邮电出版社,2006,20-23.
[2]徐青,张艳,耿则勋等.遥感影像融合与分辨率增强技术.北京,科学出版社,2007,15-18.
[3]林立宇,张友焱,孙涛等.Contourlet变换——影像处理应用.北京,科学出版社,2008,17-19.
[4]R.Y.Tsai and T.S.Huang. Multi-frame LR images resolution and registration. Advances in Computer Vision and Image Processing,1984,1:317～319.
[5]N.Nguyen and P.Milanfar. A wavelet-based interpolation restoration method for image sequence super-resolution. Circuit Systems Signal Process,2000,19(4):321-338.
[6]S.Lertrattanapanich. Super-resolution techniques from degraded image data fusion using the spatial tessellations and wavelets method. The Pennsylvania State University,2003.
[7]C.A.Segall, A.K.Katsaggelos, R.Molina and J.Matros. Bayesian resolution enhancement method of the compressed video. IEEE Transaction on Image Processing,2004,13(7):898-911.
[8]P.Vandewalle, L.Sbaiz, J.Vandewalle and M.Vetterli. Super-resolution from unregistered and totally aliased signals using subspace methods. IEEE Transactions on Signal Processing,2007,55(7):3687-3703.
[9]沈焕峰,李平湘,张良培等.超分辨率图像重建技术与方法综述.光学技术,2009,35(2)：194-199.
[10]李桐.超分辨率图像重建技术.哈尔滨师范大学自然科学学报,2006,22(5)：69-71.
[11]沈焕峰,艾廷华,刘耀林等.超分辨率图像重建技术的发展与应用现状.测控技术,2009,28(6)：5-11.
[12]李平湘,沈焕峰,张良培.超分辨率图像重建技术在遥感中的应用.地理空间信息,2007,5(5)：1-3.
[13]赵荣椿.超分辨率图像重建及其应用.测控技术,2007,26：1-7.
[14]浦剑,张军平,黄华.超分辨率算法研究综述.山东大学学报,2009,39(1)：27-32.
[15]王晓文,刘雨.超分辨率图像研究综述.信息技术,2009,(7)：236-240.
[16]苏秉华,金伟其,牛丽红等.基于Markov场的最大后验概率超分辨率图像复原法.电子学报,2002,31(4)：492-496.
[17]曹玉珍,刘晓婷,王维,等.基于凸集投影与小波融合的图像超分辨率重建方法.生物医学工程杂志,2009,26(5)：947-952.
[18]李淑静,邵峰晶.基于小波稳健的超分辨率图像重建算法.微型机与应用,2007,(6)：91-93.
[19]陶洪久,柳健,田金文.基于小波变换和插值处理的超分辨率图像处理算法.武汉理工大学学报,2002,24(8)：63-66.
[20]刘涛.序列图像的超分辨率重建技术研究.中北大学,2008.
[21]朱福珍,李金宗,李冬冬.基于BP神经网络的超分辨率图像重建.系统工程与电子技术,2009,31(7)：1746-1749.
[22]S.C.Park, M.K.Park, M.GKang. Super-resolution image reconstruction techniques:A technical overview. IEEE signal processing magazine,2003, (5):21～36.
[23]M.Elad and A.Feuer. Restoration of a single super-resolution image from several blurred, noisey, and undersampled measured images sequence. IEEE Transactions on Image Processing.1997,6(12):1646-1658.
[24]N.Nguyen, P.Milanfar and G.Golub. A computationally efficient super-resolution image reconstruction algorithm. IEEE Trans. Image Processing,2001,10:573-583.
[25]R.C.Hardie, K.J.Barnard, J.GBognar, et al.High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system. Opt. Eng.1998,37(1):247-260.
[26]A.M.Tekalp. Digital Video Processing. Englewood Cliffs, NJ:Prentice Hall,1995, 66-78.
[27]M.Irai and S.Peleg. Improving resolution by image registration. CVGIP:Graphical Models and Image Proc,1991,53:231-239.
[28]A.K.Katsaggelos, Ed. Digital Image Restoration. Heidelberg, Germany: Springer-Verlag., Springer.1991,23.
[29]R.R.Schultz and R.L.Stevenson. Extraction of the high-resolution frames from the video Sequences. IEEE Transactions on Image processing,1996,5(6):996-1101.
[30]Chiang M C, Boult T E. Efficient super resolution via image warping. Image and Vision Computing,2000,10(18):761-771.
[31]张弛,杜明辉.自适应超分辨率图像重建.计算机工程设计,2005,26(8)：2033-2035.
[32]Lertrattanapanich S, Bose N K. High resolution image formation from low resolution frames using Delaunay triangulation. IEEE Transactions on image processing,2002, 11(12):1427-1441.
[33]Baker S, Kanade T. Limits on super resolution and how to break them. IEEE Conf. Computer Vision and Pattern Recognition,2000,9(2):372-379.
[34]Hertzmann A, Analogies A. Computer graphics. New York Siggtaph, ACM Press, 2001:327-340.
[35]William T, Freeman, Jones T R, et al. Example based super resolution. IEEE Computer Graphics and Application,2002,22(2):56-65.
[36]Lee E S, Kang M G, Regularized adaptive high resolution image reconstruction considering inaccurate subpixel registration. IEEE Transaction on Image Processing, 2003,12(7):827-837.
[37]Deepu Rajan, Subhasis Chaudhuri. Generalized interpolation and its application in super-resolution considering imaging. Image and Vision Computing,2001,(19): 957-969.
[38]刘卫光,李跃,张修社等.图像信息融合与识别.北京,电子工业出版社,2008,54-67.
[39]章毓晋.图像处理.北京,清华大学出版社,2006,89-100.
[40]仲崇丽.数字图像去噪方法的比较与研究.中国新技术新产品,2010,15：41
[41]赵继印,李先涛,赵静荣,等.基于半软阈值的图像小波域去噪方法.大庆石油学院学报,2004,1(28)：63-66.
[42]Donoho D L. De-noising by soft-thresholding. IEEE Transaction on Information Theory,1995,41:613-627
[43]Y.Altunbasak, A.J.Patti and R.M.Mersereau. Super-resolution still and video rezolution from MPEG-coded video. IEEE Transaction On cricuit and Systems for video technology.2002,12(4):217-226.
[44]禹晶,苏开娜.块运动估计的研究进展.中国图象图形学报,2007,12(12)：2031-2041
[45]田胜军.基于块匹配算法的运动估计研究田胜军.电子科技大学,2006.
[46]吴庆伟,王远翔,王宏远等.序列图像相关性分析与自适应运动估计.有线电视技术,2006,(1)：37-40.
[47]陈书海,傅录祥.实用数字图像处理.北京,科学出版社,2007,103-110.
[48]张德丰,赵书梅,刘国希.MATLAB图形与动画设计.北京,国防工业出版社,2009,78-85.
[49]唐荣锡,汪嘉业,彭群生等.计算机图形学教程.北京,科学出版社,2000,89-95.
[50]吴伟仁,闫磊,田金文.基于小波变换与Bezier曲线插值相结合的图像插值方法.哈尔滨工业大学学报,2004,36(11)：1468-1470
[51]孙庆杰,张晓鹏,吴恩华.一种基于Bezier插值曲面的图像放大方法.软件学报,1999,10(2)：570-574
[52]田岩,田金文,柳健等.超分辨率技术的实现——一种改善的小波插值方法.中国图象图形学报,2003,8A(12)：1422-1426
[53]Amir Said, William A. Pearlman. Image compression using the spatial-orientation tree. Proc. IEEE Int. Symp. On Circuits and Systems, Chicago, IL,.1993:279-282.
[54]石峻,郭宝龙.一种新的基于图像插值方案——子带插值.西安电子科技大学学报,1998,25(5)：684-688
[55]赵文蕾.基于互有位移序列图像的超分辨率复原技术研究.燕山大学,2006.
[56]林虹,李庆辉,樊松波.简单有效的序列图像超分辨率重建算法.计算机应用研究.2005,9：152-153.
[57]孙婧.在小波域的基于YUV模型的彩色超分辨率图像重构.上海交通大学,2008.
[58]童佳奇.超分辨率彩色图像重建技术的研究.上海交通大学,2010.
[59]赵书斌,张蓬,彭思龙.基于小波域中HMT模型的超分辨率彩色图像重建.中国图象图形学报,2004,9(2)：172-177.
[60]雷丽明.超分辨率图像重构算法的研究.哈尔滨理工大学,2008.
[2]徐青,张艳,耿则勋等.遥感影像融合与分辨率增强技术.北京,科学出版社,2007,15-18.
[3]林立宇,张友焱,孙涛等.Contourlet变换——影像处理应用.北京,科学出版社,2008,17-19.
[4]R.Y.Tsai and T.S.Huang. Multi-frame LR images resolution and registration. Advances in Computer Vision and Image Processing,1984,1:317～319.
[5]N.Nguyen and P.Milanfar. A wavelet-based interpolation restoration method for image sequence super-resolution. Circuit Systems Signal Process,2000,19(4):321-338.
[6]S.Lertrattanapanich. Super-resolution techniques from degraded image data fusion using the spatial tessellations and wavelets method. The Pennsylvania State University,2003.
[7]C.A.Segall, A.K.Katsaggelos, R.Molina and J.Matros. Bayesian resolution enhancement method of the compressed video. IEEE Transaction on Image Processing,2004,13(7):898-911.
[8]P.Vandewalle, L.Sbaiz, J.Vandewalle and M.Vetterli. Super-resolution from unregistered and totally aliased signals using subspace methods. IEEE Transactions on Signal Processing,2007,55(7):3687-3703.
[9]沈焕峰,李平湘,张良培等.超分辨率图像重建技术与方法综述.光学技术,2009,35(2)：194-199.
[10]李桐.超分辨率图像重建技术.哈尔滨师范大学自然科学学报,2006,22(5)：69-71.
[11]沈焕峰,艾廷华,刘耀林等.超分辨率图像重建技术的发展与应用现状.测控技术,2009,28(6)：5-11.
[12]李平湘,沈焕峰,张良培.超分辨率图像重建技术在遥感中的应用.地理空间信息,2007,5(5)：1-3.
[13]赵荣椿.超分辨率图像重建及其应用.测控技术,2007,26：1-7.
[14]浦剑,张军平,黄华.超分辨率算法研究综述.山东大学学报,2009,39(1)：27-32.
[15]王晓文,刘雨.超分辨率图像研究综述.信息技术,2009,(7)：236-240.
[16]苏秉华,金伟其,牛丽红等.基于Markov场的最大后验概率超分辨率图像复原法.电子学报,2002,31(4)：492-496.
[17]曹玉珍,刘晓婷,王维,等.基于凸集投影与小波融合的图像超分辨率重建方法.生物医学工程杂志,2009,26(5)：947-952.
[18]李淑静,邵峰晶.基于小波稳健的超分辨率图像重建算法.微型机与应用,2007,(6)：91-93.
[19]陶洪久,柳健,田金文.基于小波变换和插值处理的超分辨率图像处理算法.武汉理工大学学报,2002,24(8)：63-66.
[20]刘涛.序列图像的超分辨率重建技术研究.中北大学,2008.
[21]朱福珍,李金宗,李冬冬.基于BP神经网络的超分辨率图像重建.系统工程与电子技术,2009,31(7)：1746-1749.
[22]S.C.Park, M.K.Park, M.GKang. Super-resolution image reconstruction techniques:A technical overview. IEEE signal processing magazine,2003, (5):21～36.
[23]M.Elad and A.Feuer. Restoration of a single super-resolution image from several blurred, noisey, and undersampled measured images sequence. IEEE Transactions on Image Processing.1997,6(12):1646-1658.
[24]N.Nguyen, P.Milanfar and G.Golub. A computationally efficient super-resolution image reconstruction algorithm. IEEE Trans. Image Processing,2001,10:573-583.
[25]R.C.Hardie, K.J.Barnard, J.GBognar, et al.High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system. Opt. Eng.1998,37(1):247-260.
[26]A.M.Tekalp. Digital Video Processing. Englewood Cliffs, NJ:Prentice Hall,1995, 66-78.
[27]M.Irai and S.Peleg. Improving resolution by image registration. CVGIP:Graphical Models and Image Proc,1991,53:231-239.
[28]A.K.Katsaggelos, Ed. Digital Image Restoration. Heidelberg, Germany: Springer-Verlag., Springer.1991,23.
[29]R.R.Schultz and R.L.Stevenson. Extraction of the high-resolution frames from the video Sequences. IEEE Transactions on Image processing,1996,5(6):996-1101.
[30]Chiang M C, Boult T E. Efficient super resolution via image warping. Image and Vision Computing,2000,10(18):761-771.
[31]张弛,杜明辉.自适应超分辨率图像重建.计算机工程设计,2005,26(8)：2033-2035.
[32]Lertrattanapanich S, Bose N K. High resolution image formation from low resolution frames using Delaunay triangulation. IEEE Transactions on image processing,2002, 11(12):1427-1441.
[33]Baker S, Kanade T. Limits on super resolution and how to break them. IEEE Conf. Computer Vision and Pattern Recognition,2000,9(2):372-379.
[34]Hertzmann A, Analogies A. Computer graphics. New York Siggtaph, ACM Press, 2001:327-340.
[35]William T, Freeman, Jones T R, et al. Example based super resolution. IEEE Computer Graphics and Application,2002,22(2):56-65.
[36]Lee E S, Kang M G, Regularized adaptive high resolution image reconstruction considering inaccurate subpixel registration. IEEE Transaction on Image Processing, 2003,12(7):827-837.
[37]Deepu Rajan, Subhasis Chaudhuri. Generalized interpolation and its application in super-resolution considering imaging. Image and Vision Computing,2001,(19): 957-969.
[38]刘卫光,李跃,张修社等.图像信息融合与识别.北京,电子工业出版社,2008,54-67.
[39]章毓晋.图像处理.北京,清华大学出版社,2006,89-100.
[40]仲崇丽.数字图像去噪方法的比较与研究.中国新技术新产品,2010,15：41
[41]赵继印,李先涛,赵静荣,等.基于半软阈值的图像小波域去噪方法.大庆石油学院学报,2004,1(28)：63-66.
[42]Donoho D L. De-noising by soft-thresholding. IEEE Transaction on Information Theory,1995,41:613-627
[43]Y.Altunbasak, A.J.Patti and R.M.Mersereau. Super-resolution still and video rezolution from MPEG-coded video. IEEE Transaction On cricuit and Systems for video technology.2002,12(4):217-226.
[44]禹晶,苏开娜.块运动估计的研究进展.中国图象图形学报,2007,12(12)：2031-2041
[45]田胜军.基于块匹配算法的运动估计研究田胜军.电子科技大学,2006.
[46]吴庆伟,王远翔,王宏远等.序列图像相关性分析与自适应运动估计.有线电视技术,2006,(1)：37-40.
[47]陈书海,傅录祥.实用数字图像处理.北京,科学出版社,2007,103-110.
[48]张德丰,赵书梅,刘国希.MATLAB图形与动画设计.北京,国防工业出版社,2009,78-85.
[49]唐荣锡,汪嘉业,彭群生等.计算机图形学教程.北京,科学出版社,2000,89-95.
[50]吴伟仁,闫磊,田金文.基于小波变换与Bezier曲线插值相结合的图像插值方法.哈尔滨工业大学学报,2004,36(11)：1468-1470
[51]孙庆杰,张晓鹏,吴恩华.一种基于Bezier插值曲面的图像放大方法.软件学报,1999,10(2)：570-574
[52]田岩,田金文,柳健等.超分辨率技术的实现——一种改善的小波插值方法.中国图象图形学报,2003,8A(12)：1422-1426
[53]Amir Said, William A. Pearlman. Image compression using the spatial-orientation tree. Proc. IEEE Int. Symp. On Circuits and Systems, Chicago, IL,.1993:279-282.
[54]石峻,郭宝龙.一种新的基于图像插值方案——子带插值.西安电子科技大学学报,1998,25(5)：684-688
[55]赵文蕾.基于互有位移序列图像的超分辨率复原技术研究.燕山大学,2006.
[56]林虹,李庆辉,樊松波.简单有效的序列图像超分辨率重建算法.计算机应用研究.2005,9：152-153.
[57]孙婧.在小波域的基于YUV模型的彩色超分辨率图像重构.上海交通大学,2008.
[58]童佳奇.超分辨率彩色图像重建技术的研究.上海交通大学,2010.
[59]赵书斌,张蓬,彭思龙.基于小波域中HMT模型的超分辨率彩色图像重建.中国图象图形学报,2004,9(2)：172-177.
[60]雷丽明.超分辨率图像重构算法的研究.哈尔滨理工大学,2008.