基于正则化MAP方法的图像超分辨率重建
|
摘要
超分辨率(super-resolution)重建是通过图像处理技术从多幅低分辨率图像(low-resolution)中获得一幅高分辨率图像(high-resolution)的过程。已有文献一般在假设位移已知或点扩展函数(PSF)已知的情况下重建SR图像,但很少涉及两者都未知的情况。可在实际情况中,能够得到的往往只是同一场景的多幅图像数据,而位移和PSF都是未知的。在这种情况下,如何重建出一幅高分辨率图像便是本论文要解决的问题。本论文提出了一种新型实用的基于未知位移和未知PSF的SR重建方法。我们首先利用LR (low-resolution)图像帧内像素点的相关性,再利用块匹配算法获得亚像素点位移估计。然后在选择了ML算法进行盲解卷积。本论文最大的特点是除了需要低分辨率图像数据外,不需要额外的先验知识,而且提出的解决SR重建问题的方法在位移估计和盲解卷积上采用了简单易用的方法,整体方法计算复杂度低,实用性强。在仿真结果上,得到了较好的重建效果。
由于SR重建问题往往并不是由一个单一算法就能够解决的问题,常常是需要一个整体的解决方案,因此,为了获得良好的重建效果,本论文详细地分析比较了各种可能用于SR重建问题的方法,具体的如统计方法,图像处理方法,优化算法等,其中对于运动估计,比较了常用的几类估计方法,从中选择了块匹配算法。对于盲解卷积方法,同样比较了几种可用的方法,并从中选择了一种计算简单,应用方便的盲解卷积方法。
此外,为了更好地适应于快速SR重建问题,我们也对一些方法进行了改进,如在运动估计和盲解卷积方法上,我们对相应算法做了改进,使其能在计算上效率更高,使重建效果更好。
SR (super-resolution) reconstruction is a restoring method which makes use of a series of LR images to restore an HR image based on image processing technology. Several common SR methods are analyzed. In literature, the motion or PSF (point spread function) is generally supposed as known. However, less study considers the case that both of them are unknown. However, in a real situation, there is only image data and the motion and PSF are unknown. In this situation, how to reconstruct HR image is the key problem the paper deals with. In this paper, a useful SR reconstruction method for unknown global translation and unknown PSF is proposed. For motion estimation, the relation of pixels in an LR (low-resolution) image is used to realize the sub-pixel translation estimation and for specific estimation algorithm, the easily used block-matching algorithm is chosen. For unknown PSF, the necessity of blind deconvolution is discussed. Since ML algorithm is less constraint, it is chosen for blind deconvolution. The outstanding characteristic of the paper is that the only necessity for SR construction is the LR image data showing the same scene. Besides, the SR method proposed is easily applied with less computing complexity. The simulation results have been presented to show the effectiveness of the proposed algorithm.
Since the problem of SR reconstruction is unable to be solved by one method, an overall approach is usually needed. Thus in order to improve the restoring efficiency and restoring effect, a lot of methods which are able to be used in SR reconstruction are discussed and compared, such as statistical methods, image processing methods and optimization algorithm etc. Specifically for motion estimation, block matching method is chosen after several motion estimation methods discussed and for blind deconvolution method, a convenient method is chosen from several blind deconvolution methods.
Besides, several existing methods are improved in order to reconstruct HR image faster and better. For example, in motion estimate and blind deconvolution, we have improved the existing algorithms, which enhance the efficiency of reconstruction.
由于SR重建问题往往并不是由一个单一算法就能够解决的问题,常常是需要一个整体的解决方案,因此,为了获得良好的重建效果,本论文详细地分析比较了各种可能用于SR重建问题的方法,具体的如统计方法,图像处理方法,优化算法等,其中对于运动估计,比较了常用的几类估计方法,从中选择了块匹配算法。对于盲解卷积方法,同样比较了几种可用的方法,并从中选择了一种计算简单,应用方便的盲解卷积方法。
此外,为了更好地适应于快速SR重建问题,我们也对一些方法进行了改进,如在运动估计和盲解卷积方法上,我们对相应算法做了改进,使其能在计算上效率更高,使重建效果更好。
SR (super-resolution) reconstruction is a restoring method which makes use of a series of LR images to restore an HR image based on image processing technology. Several common SR methods are analyzed. In literature, the motion or PSF (point spread function) is generally supposed as known. However, less study considers the case that both of them are unknown. However, in a real situation, there is only image data and the motion and PSF are unknown. In this situation, how to reconstruct HR image is the key problem the paper deals with. In this paper, a useful SR reconstruction method for unknown global translation and unknown PSF is proposed. For motion estimation, the relation of pixels in an LR (low-resolution) image is used to realize the sub-pixel translation estimation and for specific estimation algorithm, the easily used block-matching algorithm is chosen. For unknown PSF, the necessity of blind deconvolution is discussed. Since ML algorithm is less constraint, it is chosen for blind deconvolution. The outstanding characteristic of the paper is that the only necessity for SR construction is the LR image data showing the same scene. Besides, the SR method proposed is easily applied with less computing complexity. The simulation results have been presented to show the effectiveness of the proposed algorithm.
Since the problem of SR reconstruction is unable to be solved by one method, an overall approach is usually needed. Thus in order to improve the restoring efficiency and restoring effect, a lot of methods which are able to be used in SR reconstruction are discussed and compared, such as statistical methods, image processing methods and optimization algorithm etc. Specifically for motion estimation, block matching method is chosen after several motion estimation methods discussed and for blind deconvolution method, a convenient method is chosen from several blind deconvolution methods.
Besides, several existing methods are improved in order to reconstruct HR image faster and better. For example, in motion estimate and blind deconvolution, we have improved the existing algorithms, which enhance the efficiency of reconstruction.
引文
[1]TSAI R Y, HUANG T S. Multiframe image restoration and registration[J]. Advances in Computer Vision and Image Processsing,1984(1):317-339.
[2]BOSE N K, VALENZUELA H M. Recursive reconstructionof high resolution image from noisy undersampled multiframes[J]. IEEE Trans. on Accoustics, Speech and Signal Processing, 1990,18(6):1013-1027.
[3]KIM S P, SU W Y. Recursive high-resolution reconstructionof blurred multiframe images [J]. IEEE Trans. on Image Processing,1993,2:534-539.
[4]IRANI M, PELEG S. Improving resolution by image registration [J]. CVGIP:Graphical Models and Image Processing,1991,53:231-239.
[5]STARK H, OSKUI P. High-resolution image recovery from imageplanearrays using convex projections[J], J. Optical Society of America,1989,6(11):1715-1726.
[6]TOM B C, KATSAGGELOS A K. Reconstruction of ahigh-resolution image by simultaneous registration, restoration and interpolation of low-resolution images[C]. Proc. of Int Conf. Image Processing, Washington D.C:IEEE,1995:539-542.
[7]HARDIE R C, BARNARD K J, ARMSTRONG E E. Joint MAP registration and high resolution image estimation using a sequence of undersampled images[J]. IEEE Trans. on Image Processing,1997,6(12):1621-1633.
[8]ELAD M, FEUER A. Restoration of a single superresolutionimage from several blurred, noisy and undersampled measured images[J]. IEEE Trans on Image Processing,1997,6(12): 1646-1658.
[9]MARTINS A L D, LEVADA A L M, HOMEM M R P, et al. Map-MRF super-resolution image reconstruction using maximum pseudo-likelihood parameter estimation[C]. IEEE International Conference on Image Processing (ICIP) 2009, Cario:IEEE 2009.
[10]BABACAN S D, MOLINA R, KATSAGGELOS A K. Variational Bayesian super resolution[J]. IEEE Trans on Image Processing,2011,20(4):984-999.
[11]NGUYEN N, MILANFAR P, GOLUB G. Efficient generalized cross-validation with applications to parametric image restoration andresolution enhancement[J]. IEEE Trans on Image Processing,2001,10(9):1299-1308.
[12]NGUYEN N, MILANFAR P, GOLUB G. A computationally efficient superresolutionimage reconstruction algorithm[J]. IEEE Trans on Image Processing,2001,10:573-583.
[13]MOLINA R, KATSAGGELOS A K, MATEOS J. Bayesian and regularizationmethods for hyperparameter estimation in image restoration [J]. IEEE Trans on Image Processing,1999, 8(2):231-246.
[14]BAKER S, KANADE T. Limits on super-resolution and how to break them[J]. IEEE Trans on Pattern Analysis and Machine Intelligence,2002,24(9):1167-1183.
[15]SROUBEK F, CRISTOBAL G. A unified approach to superresolution and multichannel blind deconvolution[J]. IEEE Trans on Image Processing,2007,16(9):2322-2332.
[16]FARSIU S, ROBINSON M D, ELAD M, et al. Fast and robust multiframe super resolution[J]. IEEE Trans on Image Processing,2004,13(10):1327-1344.
[17]ZOMET A, PELEG S. Efficient super-resolution and applications tomosaics[C]. Proc. Int. Conf. Pattern Recognition, Barcelona:IEEE,2000.
[18]DUFAUX F, MOSCHENI F, Motion Estimation Techniques for Digital TV:A Review and a New Contribution [J]. Proceedings of The IEEE,1995,83(6):858-876.
[19]LUCAS B D, KANADE T. An Iterative Image Registration Technique with An Applicationto Stereo Vision [C]. The 7th International International Joint Conference on Artificial Intelligence (1981) 674-679.
[19]MANIKANDAN M, VIJAYAKUMAR P, RAMADASS N. Motion estimation method for video compression:an overview[C]. IFIP International Conference on Wireless and Optical Communications Networks, Santiago:Springer,2006.
[19]BROX T, MALIK J. Large displacement optical flow:descriptormatching in variational motion estimation[J]. IEEE Trans on Pattern Analysis and Machine Intelligence,2011,33(3): 500-513.
[20]ZHU S, MA K. A new diamond search algorithm for fast block-matching motion estimation[J]. IEEE Trans on Image Processing,2002,9(2):287-290.
[21]LU J, LIOU L. A simple and efficient search algorithm for block-matching motion estimation[J]. IEEE Trans on CAS For Video Technology,1997,7(2):429-433.
[22]KAY S M. Fundamentals of Statistical Signal Processing:Estimation Theory[M]. Englewood Cliffs, NJ:Prentice-Hall,1993, chap.10-11.
[23]. MARTINS D, HONMEM P, MASCARENHAS A. SuperResolution Image Reconstruction using the Generalized Isotropic MultiLevel Logistic Model[C]. SAC' 09,2009, Hawaii:ACM, 934-938.
[26]BABACAN S D, MOLINA R, KATSAGGELOS A K. Parameter estimation in TV image restoration using variational distribution approximation[J]. IEEE Trans on Image Processing, 2008,17(3):326-339.
[27]BIOUCAS-DIAS J, FIGUEIREDO M, OLIVEIRA J. Total-variation image deconvolution:A majorization-minimization approach[C]. ICASSP, Toulouse:IEEE,2006.
[28]LANG T, PERREA S. Multichannel blind identification:from subspace to maximum likelihood methods[J]. Proceedings of the IEEE,1998,86(10):1951-1968.
[29]KUNDUR D, HATZINAKOS D. Blind image deconvolution[J]. Signal processing magazine, IEEE,1996,13(3):43-64.
[30]NOCEDAL J, WRIGHT S. Numerical Optimization 2nd Edition[M]. USA:Springer,2006.
[31]GOLUB G H, VAN LOAN C F, Matrix Computations[M]. Baltimore, MD:The Johns Hopkins University Press,1996.
[32]LAGENDIJK R L, BIEMOND J, BOEKEE D E. Regularized iterative image restoration with ringing reduction[J]. IEEE Trans on Acoustics, Speech and Signal Processing,1988, 13(12):1874-1888.
[33]SEGALL C A, KATSAGGELOS A K, MOLINA R, MATEOS J. Bayesian resolution enhancement of compressed video[J]. IEEE Trans on Image Processing,2004,13(7):898-911.
[34]DEMPSTER A P, LAIRD N M, RUBIN D B. Maximum-likelihood from incomplete data via the em algorithm[J]. J. Royal Statist. Soc. Ser. B,1977,1-38.
[35]WU C F. On the convergence properties of the em algorithm[J]. The Annals of Statistics, 1983,11(1):95-103.
[36]刘良云等,超分辨率图象重构技术的仿真实验研究[J],中国图像图形学报,2001(6)：629-630.
[37]赵书斌等,基于小波域LS方法的图象超分辨率重构算法[J],中国图像图形学报,2003(8)： 1281-1285.
[38]刘志军等,基于并行遗传算法的图像超分辨率复原[J],中国图像图形学报,2003(9)：62-68.
[39]RICHARDSON W H. Bayesian-based iterative method of image restoration.[J]. J. Opt. Soc. Am.1972 (62):55-59.
[40]LUCY L B. An iterative technique for the rectification of observed distributions [J]. Astron. J. 1974(79):745-765.
[41]FERGUS R, SINGH B, HERTZMANN A, et al. Removing camera shake from a single photograph[J]. ACM Transactions on Graphics.2006(25):787-794.
[42]SHAN Q, JIA J, AGARWALA A. High-quality motion deblurring from a single image[J]. ACM Trans. Graphics.2008(27) article no.73.
[43]CHO S, LEE S. Fast motion deblurring[J]. ACM Trans. Graphics.2009(28) article no.145
[44]FIELD D. What is the goal of sensory coding?[J] Neural Computation 1994(6):559-601.
[45]MISKIN J, MACKAY, D. Ensemble learning for blindimage separation and deconvolution[J]. Advances in Independent Component Analysis.2000.123-141.
[46]YUAN L, SUN J, QUAN L, et al. Image deblurring with blurred/noisy image pairs [J], ACM Trans. Graphics 2007(26).
[2]BOSE N K, VALENZUELA H M. Recursive reconstructionof high resolution image from noisy undersampled multiframes[J]. IEEE Trans. on Accoustics, Speech and Signal Processing, 1990,18(6):1013-1027.
[3]KIM S P, SU W Y. Recursive high-resolution reconstructionof blurred multiframe images [J]. IEEE Trans. on Image Processing,1993,2:534-539.
[4]IRANI M, PELEG S. Improving resolution by image registration [J]. CVGIP:Graphical Models and Image Processing,1991,53:231-239.
[5]STARK H, OSKUI P. High-resolution image recovery from imageplanearrays using convex projections[J], J. Optical Society of America,1989,6(11):1715-1726.
[6]TOM B C, KATSAGGELOS A K. Reconstruction of ahigh-resolution image by simultaneous registration, restoration and interpolation of low-resolution images[C]. Proc. of Int Conf. Image Processing, Washington D.C:IEEE,1995:539-542.
[7]HARDIE R C, BARNARD K J, ARMSTRONG E E. Joint MAP registration and high resolution image estimation using a sequence of undersampled images[J]. IEEE Trans. on Image Processing,1997,6(12):1621-1633.
[8]ELAD M, FEUER A. Restoration of a single superresolutionimage from several blurred, noisy and undersampled measured images[J]. IEEE Trans on Image Processing,1997,6(12): 1646-1658.
[9]MARTINS A L D, LEVADA A L M, HOMEM M R P, et al. Map-MRF super-resolution image reconstruction using maximum pseudo-likelihood parameter estimation[C]. IEEE International Conference on Image Processing (ICIP) 2009, Cario:IEEE 2009.
[10]BABACAN S D, MOLINA R, KATSAGGELOS A K. Variational Bayesian super resolution[J]. IEEE Trans on Image Processing,2011,20(4):984-999.
[11]NGUYEN N, MILANFAR P, GOLUB G. Efficient generalized cross-validation with applications to parametric image restoration andresolution enhancement[J]. IEEE Trans on Image Processing,2001,10(9):1299-1308.
[12]NGUYEN N, MILANFAR P, GOLUB G. A computationally efficient superresolutionimage reconstruction algorithm[J]. IEEE Trans on Image Processing,2001,10:573-583.
[13]MOLINA R, KATSAGGELOS A K, MATEOS J. Bayesian and regularizationmethods for hyperparameter estimation in image restoration [J]. IEEE Trans on Image Processing,1999, 8(2):231-246.
[14]BAKER S, KANADE T. Limits on super-resolution and how to break them[J]. IEEE Trans on Pattern Analysis and Machine Intelligence,2002,24(9):1167-1183.
[15]SROUBEK F, CRISTOBAL G. A unified approach to superresolution and multichannel blind deconvolution[J]. IEEE Trans on Image Processing,2007,16(9):2322-2332.
[16]FARSIU S, ROBINSON M D, ELAD M, et al. Fast and robust multiframe super resolution[J]. IEEE Trans on Image Processing,2004,13(10):1327-1344.
[17]ZOMET A, PELEG S. Efficient super-resolution and applications tomosaics[C]. Proc. Int. Conf. Pattern Recognition, Barcelona:IEEE,2000.
[18]DUFAUX F, MOSCHENI F, Motion Estimation Techniques for Digital TV:A Review and a New Contribution [J]. Proceedings of The IEEE,1995,83(6):858-876.
[19]LUCAS B D, KANADE T. An Iterative Image Registration Technique with An Applicationto Stereo Vision [C]. The 7th International International Joint Conference on Artificial Intelligence (1981) 674-679.
[19]MANIKANDAN M, VIJAYAKUMAR P, RAMADASS N. Motion estimation method for video compression:an overview[C]. IFIP International Conference on Wireless and Optical Communications Networks, Santiago:Springer,2006.
[19]BROX T, MALIK J. Large displacement optical flow:descriptormatching in variational motion estimation[J]. IEEE Trans on Pattern Analysis and Machine Intelligence,2011,33(3): 500-513.
[20]ZHU S, MA K. A new diamond search algorithm for fast block-matching motion estimation[J]. IEEE Trans on Image Processing,2002,9(2):287-290.
[21]LU J, LIOU L. A simple and efficient search algorithm for block-matching motion estimation[J]. IEEE Trans on CAS For Video Technology,1997,7(2):429-433.
[22]KAY S M. Fundamentals of Statistical Signal Processing:Estimation Theory[M]. Englewood Cliffs, NJ:Prentice-Hall,1993, chap.10-11.
[23]. MARTINS D, HONMEM P, MASCARENHAS A. SuperResolution Image Reconstruction using the Generalized Isotropic MultiLevel Logistic Model[C]. SAC' 09,2009, Hawaii:ACM, 934-938.
[26]BABACAN S D, MOLINA R, KATSAGGELOS A K. Parameter estimation in TV image restoration using variational distribution approximation[J]. IEEE Trans on Image Processing, 2008,17(3):326-339.
[27]BIOUCAS-DIAS J, FIGUEIREDO M, OLIVEIRA J. Total-variation image deconvolution:A majorization-minimization approach[C]. ICASSP, Toulouse:IEEE,2006.
[28]LANG T, PERREA S. Multichannel blind identification:from subspace to maximum likelihood methods[J]. Proceedings of the IEEE,1998,86(10):1951-1968.
[29]KUNDUR D, HATZINAKOS D. Blind image deconvolution[J]. Signal processing magazine, IEEE,1996,13(3):43-64.
[30]NOCEDAL J, WRIGHT S. Numerical Optimization 2nd Edition[M]. USA:Springer,2006.
[31]GOLUB G H, VAN LOAN C F, Matrix Computations[M]. Baltimore, MD:The Johns Hopkins University Press,1996.
[32]LAGENDIJK R L, BIEMOND J, BOEKEE D E. Regularized iterative image restoration with ringing reduction[J]. IEEE Trans on Acoustics, Speech and Signal Processing,1988, 13(12):1874-1888.
[33]SEGALL C A, KATSAGGELOS A K, MOLINA R, MATEOS J. Bayesian resolution enhancement of compressed video[J]. IEEE Trans on Image Processing,2004,13(7):898-911.
[34]DEMPSTER A P, LAIRD N M, RUBIN D B. Maximum-likelihood from incomplete data via the em algorithm[J]. J. Royal Statist. Soc. Ser. B,1977,1-38.
[35]WU C F. On the convergence properties of the em algorithm[J]. The Annals of Statistics, 1983,11(1):95-103.
[36]刘良云等,超分辨率图象重构技术的仿真实验研究[J],中国图像图形学报,2001(6)：629-630.
[37]赵书斌等,基于小波域LS方法的图象超分辨率重构算法[J],中国图像图形学报,2003(8)： 1281-1285.
[38]刘志军等,基于并行遗传算法的图像超分辨率复原[J],中国图像图形学报,2003(9)：62-68.
[39]RICHARDSON W H. Bayesian-based iterative method of image restoration.[J]. J. Opt. Soc. Am.1972 (62):55-59.
[40]LUCY L B. An iterative technique for the rectification of observed distributions [J]. Astron. J. 1974(79):745-765.
[41]FERGUS R, SINGH B, HERTZMANN A, et al. Removing camera shake from a single photograph[J]. ACM Transactions on Graphics.2006(25):787-794.
[42]SHAN Q, JIA J, AGARWALA A. High-quality motion deblurring from a single image[J]. ACM Trans. Graphics.2008(27) article no.73.
[43]CHO S, LEE S. Fast motion deblurring[J]. ACM Trans. Graphics.2009(28) article no.145
[44]FIELD D. What is the goal of sensory coding?[J] Neural Computation 1994(6):559-601.
[45]MISKIN J, MACKAY, D. Ensemble learning for blindimage separation and deconvolution[J]. Advances in Independent Component Analysis.2000.123-141.
[46]YUAN L, SUN J, QUAN L, et al. Image deblurring with blurred/noisy image pairs [J], ACM Trans. Graphics 2007(26).