快速图像超分辨率重构算法的研究
|
摘要
图像的超分辨率重构是指利用同一景物的一幅或多幅低分辨率图像来估计该景物的较高分辨率图像。在不提高图像获取设备物理分辨率的前提下,该方法能通过软件算法提高图像的分辨率。在遥感图像处理、军事侦查、医学成像、机器视觉、公共安全和多媒体电子消费等领域有着广泛的应用。
本文研究图像超分辨率重构的快速算法。首先,分析了图像退化的向量式模型,通过矩阵的张量积和矩阵空间跟向量空间的同构映射,将向量式模型转换为等价的矩阵式模型,与向量式模型相比,本文模型节省了计算所需的存储量,且节省量正比于降采样因子和原始低分辨率图像大小的乘积。然后,在矩阵式图像退化模型的基础上,利用最速下降法和共轭梯度法来求取图像超分辨率重构问题的最优解。接着,提出了一种快速迭代式投影重构算法,通过在矩阵空间中定义一个求解重构问题的投影算子,并利用高通滤波器来获取并增强图像的高频信息,以提高图像的分辨率。最后,设计了图像超分辨率重构软件,用于实现上述重构算法。
计算机仿真结果证明了所提算法的正确性,且计算速度与现有算法相比有较大的提高,可以直接应用于大尺寸图像的重构。综合考虑计算时间和重构质量,本文所研究的迭代投影算法具有较好的应用前景。
Super-resolution image reconstruction is a technique which can obtain a high resolution image from one or multiple warped, blurred and noised low resolution image(s). It can increase the spatial resolution of image by software without updating hardware system. So it has a wide application in remote sensing, military detection, medical imaging, machine vision, public security, multimedia E-commerce and etc.
This dissertation is concentrate on fast image SR algorithms. First, the matrix based image degrading model is obtained with the aid of matrix tensor product and the isomorphision that between matrix and vector space. Matrix model need less memory and analysis shows the saved memory is proportional to the product of subsampling factor and the dimension of low resolution image. Second, two fast SR algorithms called Matrix-Gradient-Descent and Matrix-Conjugate-Gradient which using the matrix as their variable are proposaled under the regular theory and matrix degrading model. Third, a projective operator is defined on the matrix space which can get the high resolution image from any other estimation directly. Combine with a Laplace high pass filter, high-frequency components can be got and boosted, this can be used to increase the spatial resolution of a image, based on this a fast iterative projection SR reconstruction algorithm is proposaled. At the end, SR software is designed to realize the proposed algorithms.
The validity of proposaled algorithms is proved by computer simulation. And, low resolution image with large dimensions can be reconstuctioned directly by these algorithms for their faster speed. The iterative projection SR algorithm has the highest cost-performance when a trade off exists between the computing time and reconstruction quality.
本文研究图像超分辨率重构的快速算法。首先,分析了图像退化的向量式模型,通过矩阵的张量积和矩阵空间跟向量空间的同构映射,将向量式模型转换为等价的矩阵式模型,与向量式模型相比,本文模型节省了计算所需的存储量,且节省量正比于降采样因子和原始低分辨率图像大小的乘积。然后,在矩阵式图像退化模型的基础上,利用最速下降法和共轭梯度法来求取图像超分辨率重构问题的最优解。接着,提出了一种快速迭代式投影重构算法,通过在矩阵空间中定义一个求解重构问题的投影算子,并利用高通滤波器来获取并增强图像的高频信息,以提高图像的分辨率。最后,设计了图像超分辨率重构软件,用于实现上述重构算法。
计算机仿真结果证明了所提算法的正确性,且计算速度与现有算法相比有较大的提高,可以直接应用于大尺寸图像的重构。综合考虑计算时间和重构质量,本文所研究的迭代投影算法具有较好的应用前景。
Super-resolution image reconstruction is a technique which can obtain a high resolution image from one or multiple warped, blurred and noised low resolution image(s). It can increase the spatial resolution of image by software without updating hardware system. So it has a wide application in remote sensing, military detection, medical imaging, machine vision, public security, multimedia E-commerce and etc.
This dissertation is concentrate on fast image SR algorithms. First, the matrix based image degrading model is obtained with the aid of matrix tensor product and the isomorphision that between matrix and vector space. Matrix model need less memory and analysis shows the saved memory is proportional to the product of subsampling factor and the dimension of low resolution image. Second, two fast SR algorithms called Matrix-Gradient-Descent and Matrix-Conjugate-Gradient which using the matrix as their variable are proposaled under the regular theory and matrix degrading model. Third, a projective operator is defined on the matrix space which can get the high resolution image from any other estimation directly. Combine with a Laplace high pass filter, high-frequency components can be got and boosted, this can be used to increase the spatial resolution of a image, based on this a fast iterative projection SR reconstruction algorithm is proposaled. At the end, SR software is designed to realize the proposed algorithms.
The validity of proposaled algorithms is proved by computer simulation. And, low resolution image with large dimensions can be reconstuctioned directly by these algorithms for their faster speed. The iterative projection SR algorithm has the highest cost-performance when a trade off exists between the computing time and reconstruction quality.
引文
1 M. G. Kang, S. Chaudhuri. Super-Resolution Image Reconstruction. IEEE Signal Processing Magazine. 2003, 20(3):19~20
2 S. C. Park, M. K. Park, and M. G. Kang. Super-resolution image reconstruction: A technical overview. IEEE Signal Processing Magazine. 2003, 20(3):21~36
3 R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong and E. A. Watson. High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system. Opt.Eng. 1998, 37(1):247~260
4叶培德. CCD应用技术的新进展.激光与红外. 1994, 20(2):5~7
5 J. S. Ronald, A. F. Michael. Regularized image reconstruction using SVD and a neural network method for matrix inversion. IEEE Transactions on signal processing. 1993, 41(10):3074~3077
6苏秉华,金伟其,牛丽红,刘广荣.超分辨率图像复原及其进展.光学技术. 2001, 2(1):6~9
7 E. Shechtman, Y. Caspi and M. Irani. Space-Time Super-Resolution. IEEE Trans. On Pattern Analysis and MAachine Intelligence. 2005, 27(4):531~545
8 J. L. Harris. Diffraction and Resolving Power. JOSA. 1964, 54(7):9312~9361
9 J. W. Goodman. Introduction to Fourier Optics. McGraw Hill, NewYork, 1968
10 H. Maitre, A. J. Levy. Superresolution Using Linear System Methods A Comparison. Acoustics, Speech, and Signal Processing, IEEE International Conference on ICASSP '82. 1982, 7:1537~1540
11 S. Farsiu, D. Robinson, M. Elad and P. Milanfar. Advances and Challenges in Super-Resolution. International Journal of Imaging Systems and Technology. 2004, 14(2):47~57
12 S. Borman and R. L. Stevenson. Super-resolution from image sequences-a review. In Proc. 1998 Midwest Symp. Circuits and Systems. 1999:374~378
13 A. Brown. Effect of Truncation on Image Enhancement by Prolate Spheroidal Function. JOSA. 1969, 59:228~229
14 A. K. Jain. Fundamentals of Digital Image Processing. Prentice-Hall, Englewood Cliffs, HJ, 1989
15 S. Wadaka, T. Sato. Superresolution in Incoherent Imaging System. JOSA. 1975, 65(3):3542~3551
16 H. C. Andrews, B. R. Hunt. Digital Image Restoration. Prentice-Hall, Englewood Cliffs , NJ. 1977
17 R. Y. Tsai and T. S. Huang. Multiframe image restoration and registration. Advances in Computer Vision and Image Processing. JAI Press Inc. 1984,1:317~339
18 B. R. Hunt. Super-Resolution of Images : Algorithms , Principles , Performance. International Journal of Imaging Systems and Technology. 1995, 6:2972~3041
19 C. K. Rusforth. In Image Reconstruction, Theory and Application. Academic Press, New York, 1987
20 W. H. Richardson. Bayesian Based Iterative Method of Image Restoration. JOSA. 1972, 62:552~601
21 B. R. Hunt and P. Sementill. Description of a Poisson Imagery Super Resolution Algorithm. In Astronomical Data Analysis Software and Systems. Astronomical Society of the Pacific, San Francisco. 1992
22 H. Stark, P. Oskoui. High Resolution Image Recovery from Image Plane Arrays, Using Convex Projections. JOSA. 1989, 6(11):1715~1726
23 A. M. Tekalp, M. K. Ozkan, M. I. Sezan. High Resolution Image Recon struction from Lower Resolution Image Sequence and Space Varying Image Restoration. In ICASSP. 1992, 3:1692~1721
24 R. R. Schultz, R. L. Stevenson. Extraction of High Resolution Frames from Video Sequences. IEEE Trans. IP. 1996, 5(6):996~1011
25 A. J. Patti, M. I. Sezan and A. M. Tekalp. Superresolution Video Reconstruction with Arbitrary Sampling Lattices and Nonzero Aperture Time. IEEE Trans. IP. 1997, 6(8):1064~1076
26 M. Elad, A. Feuer. Restoration of a Single Superresolution Image from Several Blurred, Noisy and Undersampled Measured Images. IEEE Trans. IP. 1997, 6(12):1646~1658
27 S. Zhao, H. Han and S. L. Peng. Wavelet-Domain HMT-based Image super-resolution. Image Processing, 2003. ICIP 2003. Proceedings. 2003 International Conference. 2003, 2:953~956
28 S. E. EI-Khamy, M. M. Hadhoud, M. I. Dessouky, B. M. Salam and F. E. Abd EI-Samie. Regularized super-resolution reconstruction of images using wavelet fusion. Opt. Eng. 2005, 44(9):097001
29 H. Chang, D. Y. Yeung, Y. M. Xiong. Super-Resolution Through Neighbor Embedding. CVPR’04. 2004,1:275~282
30 S. Baker and T. Kanade. Limits on Super-Resolution and How to Break Them. IEEE Trans. Patten Analysis and Machine Intelligence. 2002, 24(9):1167-1183
31 K. Farzad and S. Behzad. Robust Regularized Tomographic Imaging With Convex Projections. Image Processing, 2005. ICIP 2005. IEEE International Conference on. 2005, 2:205~208
32 P. E. Eren, M. I. Sezan and A. M. Tekalp. Robust, Object-Based High-Resolution Image Reconstruction from Low-Resolution Video. IEEE Trans. Image Processing. 1997, 6(10):1446~1451
33 B. K. Gunturk, Y. Altunbasak and R. M. Merserean. Multiframe Resolution Enhancement Methods for Compressed Video. IEEE Signal Processing Lett. 2002, 9:170~174
34 S. Farsiu, M. Elad and P. Milanfar. Multiframe Demosaicing and Super-Resolution of Color Images. IEEE Trans. Image Processing. 2006, 15(1):141~159
35 D. Rajan, S. Chaudhuri, M. V. Joshi. Multi-Objective Super Resolution: Concepts and Examples. IEEE Transactions on Signal Processing. 2003, 20(3):49~61
36陈晓丽,冯勇,龙夫年.高速TDI CCD亚像元动态成像系统分析.光学技术. 2004, 30(1):74~79
37邹谋炎.反卷积与信号复原.北京:国防工业出版社, 2001:11~24
38田岩,柳健,田金文.一种光学图像的快速超分辨率重建方法.红外与毫米波学报. 2004, 23(1):237~240
39程云鹏.矩阵论.第二版.西安:西北工业大学出版社, 2002:122~131, 276~287, 295~297, 333~343, 404~414
40张贤达.矩阵分析与应用.北京:清华大学出版社, 2004:105~108, 271~281
41 R. C. Castleman, R. E. Woods. Digital Image Processing. Publishing House of Electronics Industry, 2003:44~49
42陈宝林.最优化理论与算法.北京:清华大学出版社, 2004:334~368
43吴勃英,王德明,丁校华,李道华.数值分析原理.北京:科学出版社, 2004: 86~101
44 V. Strassen. Gaussian elimination is not optimal. Numerische Mathematik. 1969, 13:354~356
45梅生伟,申铁龙,刘康志.现代鲁棒控制理论与应用.北京:清华大学出版社, 2003:14~16
46陈省身,陈维桓.微分几何讲义.第二版.北京:北京大学出版社, 2003:1~2
47丘维声.高等代数(上).北京:高等教育出版社, 1997:133~137
48 R. C. Gonzalez, R. E. Woods.数字图像处理.第二版.阮秋琦,阮宇智.北京:电子工业出版社, 2003:98~102
49于宗义,刘希玉,闫宝强.泛函分析教程.山东大学出版社, 2003:41~53
50段广仁.线性系统理论.第二版.哈尔滨:哈尔滨工业大学出版社, 2004:153.
51 Mathworks. Matrix User’s Guide. Mathworks, 1999:4~16, 24
52 J. K. David. et al. Visual C++技术内幕.第五版.希望图书创作室.北京:北京希望电子出版社, 2006:230~248
2 S. C. Park, M. K. Park, and M. G. Kang. Super-resolution image reconstruction: A technical overview. IEEE Signal Processing Magazine. 2003, 20(3):21~36
3 R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong and E. A. Watson. High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system. Opt.Eng. 1998, 37(1):247~260
4叶培德. CCD应用技术的新进展.激光与红外. 1994, 20(2):5~7
5 J. S. Ronald, A. F. Michael. Regularized image reconstruction using SVD and a neural network method for matrix inversion. IEEE Transactions on signal processing. 1993, 41(10):3074~3077
6苏秉华,金伟其,牛丽红,刘广荣.超分辨率图像复原及其进展.光学技术. 2001, 2(1):6~9
7 E. Shechtman, Y. Caspi and M. Irani. Space-Time Super-Resolution. IEEE Trans. On Pattern Analysis and MAachine Intelligence. 2005, 27(4):531~545
8 J. L. Harris. Diffraction and Resolving Power. JOSA. 1964, 54(7):9312~9361
9 J. W. Goodman. Introduction to Fourier Optics. McGraw Hill, NewYork, 1968
10 H. Maitre, A. J. Levy. Superresolution Using Linear System Methods A Comparison. Acoustics, Speech, and Signal Processing, IEEE International Conference on ICASSP '82. 1982, 7:1537~1540
11 S. Farsiu, D. Robinson, M. Elad and P. Milanfar. Advances and Challenges in Super-Resolution. International Journal of Imaging Systems and Technology. 2004, 14(2):47~57
12 S. Borman and R. L. Stevenson. Super-resolution from image sequences-a review. In Proc. 1998 Midwest Symp. Circuits and Systems. 1999:374~378
13 A. Brown. Effect of Truncation on Image Enhancement by Prolate Spheroidal Function. JOSA. 1969, 59:228~229
14 A. K. Jain. Fundamentals of Digital Image Processing. Prentice-Hall, Englewood Cliffs, HJ, 1989
15 S. Wadaka, T. Sato. Superresolution in Incoherent Imaging System. JOSA. 1975, 65(3):3542~3551
16 H. C. Andrews, B. R. Hunt. Digital Image Restoration. Prentice-Hall, Englewood Cliffs , NJ. 1977
17 R. Y. Tsai and T. S. Huang. Multiframe image restoration and registration. Advances in Computer Vision and Image Processing. JAI Press Inc. 1984,1:317~339
18 B. R. Hunt. Super-Resolution of Images : Algorithms , Principles , Performance. International Journal of Imaging Systems and Technology. 1995, 6:2972~3041
19 C. K. Rusforth. In Image Reconstruction, Theory and Application. Academic Press, New York, 1987
20 W. H. Richardson. Bayesian Based Iterative Method of Image Restoration. JOSA. 1972, 62:552~601
21 B. R. Hunt and P. Sementill. Description of a Poisson Imagery Super Resolution Algorithm. In Astronomical Data Analysis Software and Systems. Astronomical Society of the Pacific, San Francisco. 1992
22 H. Stark, P. Oskoui. High Resolution Image Recovery from Image Plane Arrays, Using Convex Projections. JOSA. 1989, 6(11):1715~1726
23 A. M. Tekalp, M. K. Ozkan, M. I. Sezan. High Resolution Image Recon struction from Lower Resolution Image Sequence and Space Varying Image Restoration. In ICASSP. 1992, 3:1692~1721
24 R. R. Schultz, R. L. Stevenson. Extraction of High Resolution Frames from Video Sequences. IEEE Trans. IP. 1996, 5(6):996~1011
25 A. J. Patti, M. I. Sezan and A. M. Tekalp. Superresolution Video Reconstruction with Arbitrary Sampling Lattices and Nonzero Aperture Time. IEEE Trans. IP. 1997, 6(8):1064~1076
26 M. Elad, A. Feuer. Restoration of a Single Superresolution Image from Several Blurred, Noisy and Undersampled Measured Images. IEEE Trans. IP. 1997, 6(12):1646~1658
27 S. Zhao, H. Han and S. L. Peng. Wavelet-Domain HMT-based Image super-resolution. Image Processing, 2003. ICIP 2003. Proceedings. 2003 International Conference. 2003, 2:953~956
28 S. E. EI-Khamy, M. M. Hadhoud, M. I. Dessouky, B. M. Salam and F. E. Abd EI-Samie. Regularized super-resolution reconstruction of images using wavelet fusion. Opt. Eng. 2005, 44(9):097001
29 H. Chang, D. Y. Yeung, Y. M. Xiong. Super-Resolution Through Neighbor Embedding. CVPR’04. 2004,1:275~282
30 S. Baker and T. Kanade. Limits on Super-Resolution and How to Break Them. IEEE Trans. Patten Analysis and Machine Intelligence. 2002, 24(9):1167-1183
31 K. Farzad and S. Behzad. Robust Regularized Tomographic Imaging With Convex Projections. Image Processing, 2005. ICIP 2005. IEEE International Conference on. 2005, 2:205~208
32 P. E. Eren, M. I. Sezan and A. M. Tekalp. Robust, Object-Based High-Resolution Image Reconstruction from Low-Resolution Video. IEEE Trans. Image Processing. 1997, 6(10):1446~1451
33 B. K. Gunturk, Y. Altunbasak and R. M. Merserean. Multiframe Resolution Enhancement Methods for Compressed Video. IEEE Signal Processing Lett. 2002, 9:170~174
34 S. Farsiu, M. Elad and P. Milanfar. Multiframe Demosaicing and Super-Resolution of Color Images. IEEE Trans. Image Processing. 2006, 15(1):141~159
35 D. Rajan, S. Chaudhuri, M. V. Joshi. Multi-Objective Super Resolution: Concepts and Examples. IEEE Transactions on Signal Processing. 2003, 20(3):49~61
36陈晓丽,冯勇,龙夫年.高速TDI CCD亚像元动态成像系统分析.光学技术. 2004, 30(1):74~79
37邹谋炎.反卷积与信号复原.北京:国防工业出版社, 2001:11~24
38田岩,柳健,田金文.一种光学图像的快速超分辨率重建方法.红外与毫米波学报. 2004, 23(1):237~240
39程云鹏.矩阵论.第二版.西安:西北工业大学出版社, 2002:122~131, 276~287, 295~297, 333~343, 404~414
40张贤达.矩阵分析与应用.北京:清华大学出版社, 2004:105~108, 271~281
41 R. C. Castleman, R. E. Woods. Digital Image Processing. Publishing House of Electronics Industry, 2003:44~49
42陈宝林.最优化理论与算法.北京:清华大学出版社, 2004:334~368
43吴勃英,王德明,丁校华,李道华.数值分析原理.北京:科学出版社, 2004: 86~101
44 V. Strassen. Gaussian elimination is not optimal. Numerische Mathematik. 1969, 13:354~356
45梅生伟,申铁龙,刘康志.现代鲁棒控制理论与应用.北京:清华大学出版社, 2003:14~16
46陈省身,陈维桓.微分几何讲义.第二版.北京:北京大学出版社, 2003:1~2
47丘维声.高等代数(上).北京:高等教育出版社, 1997:133~137
48 R. C. Gonzalez, R. E. Woods.数字图像处理.第二版.阮秋琦,阮宇智.北京:电子工业出版社, 2003:98~102
49于宗义,刘希玉,闫宝强.泛函分析教程.山东大学出版社, 2003:41~53
50段广仁.线性系统理论.第二版.哈尔滨:哈尔滨工业大学出版社, 2004:153.
51 Mathworks. Matrix
52 J. K. David. et al. Visual C++技术内幕.第五版.希望图书创作室.北京:北京希望电子出版社, 2006:230~248