基于学习的图像超分辨率技术及其应用研究
|
摘要
超分辨率图像技术的主要目的是由提供的一幅或多幅同一场景(信息相似但细节不同)的低分辨率(Low-Resolution,LR)图像中重建高分辨率(High-Resolution,HR)图像,它克服了图像获取过程中的限制病态条件,并能获得更好的图像内容,提高场景识别的准确性。该技术广泛应用于遥感识别、图像压缩、高清电视、安全监控、视频通讯、医疗诊断、资源探测等诸多领域,是当前图像处理和计算机视觉领域中的热门研究方向之一,具有很重要的理论研究价值。
本文通过研究基于学习的图像超分辨率算法和基于梯度的图像超分辨率算法,以及图像细节增强相关方面的知识,提出了在能量优化框架下,结合边缘和细节的基于学习的单帧图像超分辨率算法。
在本文中,首先简要介绍了图像超分辨率的概念、选题背景、研究意义及近十年的国内外发展情况。其次介绍了超分辨率技术的理论基础和超分辨率在空域上的各种算法,并简要分析了各种算法的优缺点。同时用Matlab工具实现了相关算法,得到放大后的图像。接着研究了利用马尔可夫网络来学习高分辨率图像和低分辨率图像之间的关系,并通过这种关系来进行超分辨率的基于实例的图像超分辨率算法和利用图像的梯度锐化度先验知识来指导超分辨率的基于梯度的图像超分辨率,进而通过实际编程实现上述两种算法,获得了相应实验结果。通过对实验结果的理论分析,认为可以结合边缘和细节两方面来提高经过超分辨率后的图像质量,最终提出了一种新的单帧图像超分辨率算法。
本文重点介绍了能量优化框架下的基于学习的单帧图像超分辨率算法,章节3.3.1给出了超分辨率模型,并定义了能量方程和求解方法,章节3.3.2介绍了图像自身训练集的获得,章节3.3.3介绍了搜索方法,章节3.3.4给出了超分辨率算法的流程图及伪代码。从实验结果可以看出,本文提出的算法所得到的高分辨率图像在边缘和细节上都有所提高,具有更好的视觉效果。最后,对本文的研究内容进行了总结,超分辨率所面临的一些亟待解决的问题和下一步的研究重点,并对其未来的发展方向进行了展望。
The main objective of super-resolution (SR) imaging is to reconstruct a higher-resolution image based on an image or a set of images, acquired from the same scene(similar information but different details) and denoted as‘low-resolution’images. It overcomes the limitation ill-posed conditions of the image acquisition process, can get a better image content and improves a scene recognition. This technology is widely used in remote sensing recognition, image compression, high-definition television, security monitoring, video communications, medical diagnostics, resource exploration and many other fields, is one of the most popular subjects in the current field of image processing and computer vision research, and has a very important theoretical research value.
On the base of studies of the learning-based image super-resolution algorithms and the gradient-based image super-resolution algorithms, this paper proposes a learning-based single-frame image super-resolution algorithm combined with the information of edges and details under the energy optimization framework.
Firstly, we briefly introduce the concept, the background of image super-resolution and then nearly a decade of significant domestic and international developments of it in this article. Secondly, we introduce the theoretical foundation of super-resolution technology, a variety of algorithms in the spatial domain, and a brief analysis of the advantages and disadvantages of these algorithms. At the same time we use Matlab tools to achieve these correlation algorithms, and gain the enlarged images. Then we study the example-based super-resolution through a relationship of the high-resolution images and the low-resolution images learned by the Markov network and the gradient-based super-resolution algorithm using the prior knowledge of image gradient, and then by the actual programming of these two algorithms to obtain the corresponding experimental results. Through theoretical analysis of experimental results that can be combined the information of edge and detail to improve the quality of the high-resolution image. Lastly, we propose a new single-frame image super-resolution algorithm.
This paper focuses on the learning-based single-frame image super-resolution algorithm under the framework of energy optimization, section 3.3.1 gives a model of super-resolution, the definition of the energy equation and its solving method, section 3.3.2 introduces the own training set of images and section 3.3.3 the search methods, then section 3.3.4 shows the flow chart and pseudo code of the super-resolution algorithm. From the experimental results we can see that the obtained high-resolution images by the proposed algorithm are improved on the edge and detail, with a better visual. Finally, we summarize the content of this study, the problems to be solved that super-resolution has faced , the priorities of future research and prospect its future development direction.
本文通过研究基于学习的图像超分辨率算法和基于梯度的图像超分辨率算法,以及图像细节增强相关方面的知识,提出了在能量优化框架下,结合边缘和细节的基于学习的单帧图像超分辨率算法。
在本文中,首先简要介绍了图像超分辨率的概念、选题背景、研究意义及近十年的国内外发展情况。其次介绍了超分辨率技术的理论基础和超分辨率在空域上的各种算法,并简要分析了各种算法的优缺点。同时用Matlab工具实现了相关算法,得到放大后的图像。接着研究了利用马尔可夫网络来学习高分辨率图像和低分辨率图像之间的关系,并通过这种关系来进行超分辨率的基于实例的图像超分辨率算法和利用图像的梯度锐化度先验知识来指导超分辨率的基于梯度的图像超分辨率,进而通过实际编程实现上述两种算法,获得了相应实验结果。通过对实验结果的理论分析,认为可以结合边缘和细节两方面来提高经过超分辨率后的图像质量,最终提出了一种新的单帧图像超分辨率算法。
本文重点介绍了能量优化框架下的基于学习的单帧图像超分辨率算法,章节3.3.1给出了超分辨率模型,并定义了能量方程和求解方法,章节3.3.2介绍了图像自身训练集的获得,章节3.3.3介绍了搜索方法,章节3.3.4给出了超分辨率算法的流程图及伪代码。从实验结果可以看出,本文提出的算法所得到的高分辨率图像在边缘和细节上都有所提高,具有更好的视觉效果。最后,对本文的研究内容进行了总结,超分辨率所面临的一些亟待解决的问题和下一步的研究重点,并对其未来的发展方向进行了展望。
The main objective of super-resolution (SR) imaging is to reconstruct a higher-resolution image based on an image or a set of images, acquired from the same scene(similar information but different details) and denoted as‘low-resolution’images. It overcomes the limitation ill-posed conditions of the image acquisition process, can get a better image content and improves a scene recognition. This technology is widely used in remote sensing recognition, image compression, high-definition television, security monitoring, video communications, medical diagnostics, resource exploration and many other fields, is one of the most popular subjects in the current field of image processing and computer vision research, and has a very important theoretical research value.
On the base of studies of the learning-based image super-resolution algorithms and the gradient-based image super-resolution algorithms, this paper proposes a learning-based single-frame image super-resolution algorithm combined with the information of edges and details under the energy optimization framework.
Firstly, we briefly introduce the concept, the background of image super-resolution and then nearly a decade of significant domestic and international developments of it in this article. Secondly, we introduce the theoretical foundation of super-resolution technology, a variety of algorithms in the spatial domain, and a brief analysis of the advantages and disadvantages of these algorithms. At the same time we use Matlab tools to achieve these correlation algorithms, and gain the enlarged images. Then we study the example-based super-resolution through a relationship of the high-resolution images and the low-resolution images learned by the Markov network and the gradient-based super-resolution algorithm using the prior knowledge of image gradient, and then by the actual programming of these two algorithms to obtain the corresponding experimental results. Through theoretical analysis of experimental results that can be combined the information of edge and detail to improve the quality of the high-resolution image. Lastly, we propose a new single-frame image super-resolution algorithm.
This paper focuses on the learning-based single-frame image super-resolution algorithm under the framework of energy optimization, section 3.3.1 gives a model of super-resolution, the definition of the energy equation and its solving method, section 3.3.2 introduces the own training set of images and section 3.3.3 the search methods, then section 3.3.4 shows the flow chart and pseudo code of the super-resolution algorithm. From the experimental results we can see that the obtained high-resolution images by the proposed algorithm are improved on the edge and detail, with a better visual. Finally, we summarize the content of this study, the problems to be solved that super-resolution has faced , the priorities of future research and prospect its future development direction.
引文
[1] G Toraldo di Francia. Resolving power and information [J].Opt.Soc.Am.July 1955,5:497-501.
[2] Harris J L. Diffraction and resolving power [J]. Journal of the Optical Society of America, 54(7): 931-936. 1964.
[3] GOODMAN J W. Introduction to Fourier optics [M]. New York: McGraw-Hill, 1968.
[4] Brown H A. Effect of Truncation on Image Enhancement by Prolate Spheroidal Function [J]. J.O.S.A., 59: 228-229, 1969.
[5] Jain A K. Fundamentals of Digital Image Processing [M]. Prentice Hall, Englewood Cliffs. HJ, 1989.
[6] Wadaka S, Sato T. Superresolution in Incoherent Imaging System [J]. J. O. S. A., 65(3):354-355, 1975.
[7] Tsai R Y, Huang T S. Multipleframe image restoration and registration [C]. Advances in Computer Vision and Image Processing. Greenwich, USA: JAI Press Inc: 317-339. 1984
[8] Kim S P, Bose N K, and Valenzuela H M. Recursive reconstruction of high resolution image from noisy undersampled multiframes [J]. IEEE Trans. Acoust., Speech,Signal Processing, 1990,38(6):1 013-1 027.
[9] Kim S P and Su W Y. Recursive high-resolution reconstruction of blurred multiframe images [J]. IEEE Trans. Image Processing, 1993, 2(10): 534-539.
[10] Bose N K, Kim H C, and Valenzuela H M. Recursive implementation of total least squares algorithm for image reconstruction from noisy, undersampled multiframes [A]. in Proc. IEEE Conf. Acoustics, Speech and Signal Processing, Minneapolis, MN, 1993,5: 269-272.
[11] Rhee S H and Kang M G. Discrete cosine transform based regularized high-resolution image reconstruction algorithm [J]. Opt. Eng., 1999,38(8): 1348-1356.
[12] Tom, B.C., Katsaggelos, A.K.: Reconstruction of a high-resolution image by simultaneous registration, restoration, and interpolation of low-resolution images. In: Proceedings of the IEEE International Conference on Image Processing, pp. 539–542.Washington, DC (1995)
[13] Li, S.Z.: Markov Random Field Modeling in Computer Vision. Springer, New York (1995)
[14] Nguyen, N., Milanfar, P.: A wavelet-based interpolation-restoration method for super-resolution. Circuits Syst. and Signal Process.19, 321–338 (2000)
[15] W. T. Freeman, E. Pasztor, and O. Carmichael. Learning low-level vision. International Journal of Computer Vision, 40(1):25–47, 2000.
[16] Capel, D., Zisserman, A.: Super-resolution enhancement of text image sequences. In: Proceedings of the IEEE International Conference on Pattern Recognition, pp. 600–605. Barcelona, Spain(2000)
[17] Hertzmann, A., Jacobs, C.E., Oliver, N., Curless, B., Salesin, D.H.: Image analogies. In: Proceedings of the SIGGRAPH, pp.327–340. Los Angeles (2001)
[18] Morse, B., Schwartzwald, D.: Image magnification using level-set reconstruction. In: Proc. IEEEInternational Conf. Computer Vision. (2001) 333-341
[19] Liu, C., Shum, H.Y., Zhang, C.S.: Two-step approach to hallucinating faces: global parametric model and local nonparametric model. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition. (2001) 192-198
[20] Baker, S., Kanade, T.: Limits on super-resolution and how to break them. IEEE Trans. Pattern Anal. Mach. Intell. 24, 1167–1183 (2002)
[21] Tipping, M.E., Bishop, C.M. : Bayesian image super-resolution. In: Becker, S., Thrun, S., Obermeyer, K. (eds.) Advances in Neural Information Processing Systems, MIT Press, Cambridge (2002)
[22] W. Freeman, T. Jones, E. Pasztor. Example-based super-resolution. IEEE Computer Graphics and Applications, 22(2): 56-65, 2002.
[23] S.Zhao.H.Hart,S.Peng.Wavelet Domain HMT-based Image Super resolution.1EEE InL Conf.Image processing,2003.2(9):933-936
[24] Pickup, L.C., Roberts, S.J., Zisserman, A. : A sampled texture prior for image super-resolution. In: Thrun, S., Saul, L., Scholkopf, B. (eds.) Advances in Neural Information Processing Systems, MIT Press, Cambridge (2003)
[25] J. Sun, N. N. Zheng, H. Tao, H.Y. Shum. Generic Image hallucination with primal sketch prior. In CVPR, 2003.
[26] Lin, Z., Shum, H.-Y.: Fundamental limits of reconstruction-based super-resolution algorithms under local translation. IEEE Trans. Pattern Anal. Mach. Intell. 26, 83–97 (2004)
[27] Kim, K.I., Franz, M., Scholkopf, B.: Kernel hebbian algorithm for single-frame super-resolution. In: Proceedings of the ECCV Workshop on Statistical Learning in Computer Vision, pp. 135–149. Prague, Czech Republic (2004)
[28] He, H., Kondi, L.P.: Resolution enhancement of video sequences with simultaneous estimation of the regularization parameter. J. Electron. Imaging 13, 586–596 (2004)
[29] Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp.275–282. Washington, D.C. (2004)
[30] Chappalli, M.B., Bose, N.K.: Simultaneous noise filtering and super-resolution with second-generation wavelets. IEEE Signal Process. Lett. 12, 772–775 (2005)
[31] Tanaka,M., Okutomi, M.: Theoretical analysis on reconstructionbased super-resolution for an arbitrary PSF. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 947–954. San Diego, CA (2005)
[32] Pham, T.Q., van Vliet, L.J., Schutte, K.: Influence of signalto-noise ratio and point spread function on limits of superresolution. In: Proceedings of the SPIE Conference on Image Processing: Algorithms and Systems IV, pp. 169–180. San Jose, CA(2005)
[33] Robinson, D., Milanfar, P.: Statistical performance analysis of super-resolution. IEEE Trans. Image Process. 15, 1413–1428(2006)
[34] Woods, N.A., Galatsanos, N.P., Katsaggelos, A.K.: Stochastic methods for joint registration, restoration, and interpolation of multiple undersampled images. IEEE Trans. Image Process.15, 201–213 (2006)
[35] Ei-Khamy, S.E., Hadhoud, M.M., Dessouky, M.I., Salam, B.M., Ei-Samie, F.E.: Regularized super-resolution reconstruction of images using wavelet fusion. Opt. Eng. 44,097001.1–097001.10(2005)
[36] Ei-Khamy, S.E., Hadhoud, M.M., Dessouky, M.I., Salam, B.M., Ei-Samie, F.E.: Wavelet fusion: a tool to break the limits on LMMSE image super-resolution. Int. J. Wavel. Multiresolut. Inf. Process. 4, 105–118 (2006)
[37] Ji, H., Fermuller, C.: Wavelet-based super-resolution reconstruction: theory and algorithm. In: Proceedings of the European Conference on Computer Vision, pp. 295–307. Graz, Austria (2006)
[38] Bose,N.K., Ahuja, N.A.: Super resolution and noise filtering using moving least squares. IEEE Trans. Image Process. 15, 2239–2248 (2006)
[39] Pickup, L.C., Capel, D., Roberts, S.J., Zisserman, A. : Bayesian image super-resolution, continued. In: Scholkopf, B., Platt, J., Hoffman, T. (eds.) Advances in Neural Information Processing Systems, pp. 1089–1096. MIT Press, Cambridge (2006)
[40] Lin, Z.-C., He, J.-F., Tang, X.-O., Tang, C.-K.: Limits of learning-based Super-resolution algorithms. In: Proceedings of the IEEE International Conference on Computer Vision. Rio de Janeiro,Brazil (2007)
[41] Tian, J., Ma, K.-K.: Edge-adaptive super-resolution image reconstruction using a Markov chain Monte Carlo approach. In: Proceedings of the International Conference on Information, Communications and Signal Processing. Singapore (2007)
[42] Suresh, K.V., Rajagopalan, A.N.: Robust and computationally efficient super-resolution algorithm. Opt. Soc. Am. J.AOpt. Image Sci. 24, 984–992 (2007)
[43] M. K. Ng, H. Shen, L. Zhang, E. Lam, "A Total Variation Based Super-Resolution Reconstruction Algorithm for Digital Video," EURASIP Journal on Advances in Signal Processing, vol. 2007, no. Article ID 74585, 2007(SCI).
[44] Shen, H., Zhang, L., Huang, B., Li, P.: A MAP approach for joint motion estimation, segmentation, and super resolution. IEEE Trans. Image Process. 16, 479–490 (2007)
[45] Chakrabarti, A., Rajagopalan, A.N., Chellappa, R.: Super-resolution of face images using kernel PCA-based prior. IEEE Trans. Multimed. 9, 888–892 (2007)
[46] Fattal, R.: Upsampling via imposed edges statistics. ACM Trans. Graph. (Proc. of SIGGRAPH) 26 (2007)
[47] Zibetti, M., Bazan, F., Mayer, J.: Determining the regularization parameters for super-resolution problems. Signal Process. 88, 2890–2901 (2008)
[48] Katsaggelos, A.K., Molina, R.: Special issue on super resolution. Comput. J. 52, 1–167 (2008)
[49] Callico, G., Lopez, S., Sosa, O., Lopez, J.F., Sarmiento, R.: Analysis of fast block matching motion estimation algorithms for video super-resolution systems. IEEE Trans. Consum. Electron. 54, 1430–1438 (2008)
[50] Sun J, Sun J, Xu ZB, et al. Image super-resolution using gradient profile prior [C]. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2008, 2471–2478.J.
[51] Shan, Q., Li, Z., Jia, J., Tang, C.K.: Fast image/video upsampling. ACM Trans. Graph. (Proc. of SIGGRAPH ASIA) 27 (2008) 1{7}
[52] Ji, H., Fermuller, C.: Robust wavelet-based super-resolution reconstruction: theory and algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 31, 649–660 (2009)
[53] Tian, J., Ma, K.-K.: A state-space super-resolution approach for video reconstruction. Signal Image Video Process. 3, 217–240 (2009)
[54] Omer O A,Tanaka T. Region-based weighted-norm approach to video super-resolution with adaptive regularization [C] 2009 IEEE International conference on acoustics,speech,and signal processing. Taipei,Taiwan:[s.n.], 2009:833-836.
[55] Protter, M., Elad, M.: Super resolution with probabilistic motion estimation. IEEE Trans. Image Process. 18, 1899–1904(2009)
[56] Zhang, Y., Zhao, D., Ji, X.,Wang, R., Gao,W.: A spatio-temporal auto regressive model for frame rate up conversion. IEEE Trans. Circuits Syst. Video Technol. 19, 1289–1301 (2009)
[57] Glasner D,Bagon S,Irani M. Super-Resolution from a Single Image [C] IEEE 12th International Conference on computer vision. Kyoto,Japan:[s.n.], 2009:349-456.
[58] H. Takeda, P. Milanfar, M. Protter, M. Elad, Super-Resolution Without Explicit Subpixel Motion Estimation, IEEE transactions on image processing, 18 (9) (2009) pp. 1958-1975
[59] M. Protter, M. Elad, H. Takeda, and P. Milanfar. Generalizing the Non-Local-Means to Super-Resolution Reconstruction. IEEE Transactions on Image Processing, Vol. 18, No. 1, Pages 36-51, January 2009.
[60] Yang, J.,Wright, J., Huang, T.,Ma,Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19, 2861–2873(2010)
[61] Belekos, S., Galatsanos, N.P., Katsaggelos, A.K.: Maximum a posteriori video super-resolution using a new multichannel image prior. IEEE Trans. Image Process. 19, 1451–1464 (2010)
[62] Tian, J., Ma,K.-K.: Stochastic super-resolution image reconstruction. J. Vis. Commun. Image Represent. 21, 232–244 (2010)
[63] Shuaicheng Liu, Michael S. Brown, Seon Joo Kim, Yu-Wing Tai. Colorization for Single Image Super Resolution. In Proceedings of ECCV (6)'2010. pp.323~336
[64] Yu-Wing Tai, Shuaicheng Liu, Michael S B and Stephen Lin.: Super resolution using edge prior and single image detail synthesis. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition. (2010) 1{8}
[65] Agrawal A,Gupta M,Veeraraghavan A,et al. Optimal Coded Sampling for Temporal Super-Resolution [C] 2010 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). San Francisco,CA:[s.n.], 2010:599-606.
[66] Zibettia, M.V.W., Baznb, F.S.V., Mayer, J.: Estimation of the parameters in regularized simultaneous super-resolution. Pattern Recognit. Lett. 32, 69–78 (2011)
[67] H. Zimmer and A. Bruhn and J. Weickert }, Freehand HDR Imaging of Moving Scenes with Simultaneous Resolution Enhancement, Computer Graphics Forum (Proceedings of Eurographics) 30,2,405—414(2011)
[68] O. Shahar, A. Faktor and M. Irani, Space-Time Super-Resolution from a Single Video. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2011.
[69] H. Hou and H. Andrews,“Cubic splines for image interpolation and digital filtering,”IEEE Trans. Acoust., Speech, Signal Process., vol. 26, no. 6, pp. 508–517, Dec. 1978.
[70] M Irani, S Peleg. Super resolution from image sequences [A]. In Proceedings of International Conference on Pattern Recognition. Piscataway, NJ, USA.1990:115-120.
[71] M Irani, S Peleg. Improving resolution by image registration [J]. Graph.Models Image Process,1991, 5(53):231-239.
[72] M Irani, S Peleg. Motion analysis for image enhancement. Resolution, occlusion, and transparency [J]. J Vis. Commun. Image, 1993, 4(4):324-336.
[73] S Mann, R W Picard. Virtual bellows: Constructing high quality stills from Video [C]. IEEE Int. Conf. Image Proc. Austin, TX,1994:13-16.
[74] B. C. Tom, A. K. Katsaggelos. Resolution enhancement of video sequences using motion compensation. In: Proc IEEE Int Conf Image Processing. Lausanne, Switzerland. 1996,1:713-716
[75] B. C. Tom, A. K. Katsaggelos. Resolution enhancement of color video. In: Proc Conf Eur Signal Processing. 1996. 1:145-148
[76] SCHULTZ R R.STEVENSON R L. Improved definition video frame enhancement. 1995
[77] R. C. Hardie, K. J. Barnard, and E. E. Armstrong. Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Transactions on Image Processing, 6(12):1621–1633, December, 1997.
[78] Sean Borman and Robert L. Stevenson. Image Resampling and Constraint Formulation for Multi-Frame Super-Resolution Restoration. 5016:208-219, San Jose, CA, USA, Jan, 2003.
[79] H Stark, P Oskoui. High resolution image recovery from image-plane arrays, using convex projections [J]. J. Opt. Soc. Am. A, 1989, 6:1715-1726。
[80] A J Patti,M I Sezan,A M Tekalp. Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time [J]. IEEE Trans. Image Proc. , 1997, 8(6):1064-1076.
[81] P E Eren, M I Sezan, A M Tekalp. Robust, object-based high-resolution image reconstruction from low-resolution video [J]. IEEE Trans. Image Proc. , 1997, 10(6): 1446-1451.
[82] V. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
[83] N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines. New York: Cambridge Univ. Press, 2000.
[84] M. Palaniswami and A. Shilton,“Adaptive support vector machines for regression,”presented at the 9th Int. Conf. Neural Information Processing, 2002.
[85] Nguyen N,MilanFar P,Golub G.Efficient generalized cross validation with application to parametric image restoration and resolution enhancement [J].IEEE Transaction on Image Processing,2001.10(9):1299—1308
[86] Yu-Wing Tai and Shuaicheng Liu and Michael S. Brown and Stephen Lin. Super Resolution using Edge Prior and Singale Image Detail Synthesis, CVPR, 2010
[87] Zeev Farbman, Raanan Fattal, Dani Lischinski, Richard Szeliski. Edge-Preserving Decompositions for Multi-Scale Tone and Detail Manipulation. ACM Transactions on Graphics 27:3,2008
[88] Freedman, Gilad and Fattal, Raanan. Image and Video Upscaling from Local Self-Examples , ACM Trans. Graph. 28(3): 0730-0301 1-10,2010
[89] Allebach and P. W. Wong,Edge-directed interpolation. In Proc. IEEE Int. Conf. Image Processing, 1996,3:707–710.
[90] Xin Li, Michael T.Orchard.New Edge-Directed Interpolation.IEEE TRANSACTIONS ON IMAGE PROCESSING, OCTOBER 2001,10(10).
[91] Nira Shezaft, Hagit Abramov-Segalt, Ilan Sutskovert and Ran Bar-Sella.Adaptive Low Complexity Algorithm For Image Zooming At Fractional Scaling Ratio.IEEE,2000.
[92]吴良武和欧宗英.保持轮廓清晰光滑的灰度图像放大算法.计算机辅助设计与图像学学报,2002,04,14(4).
[93] Osher and Sethian. Fronts propogating with curvature-dependent speed:Algorithms based on Hamilton-Jacobi formulations , Journal of Computational Physics 79 , 1988,12-49.
[94] Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans Pattern Anal Mach Intell 23 (2001) 1222-1239
[95] Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts. IEEE Trans Pattern Anal Mach Intell 26 (2004) 147-59
[96] Yedidia, J.S., Freeman, W.T., Weiss, Y.: Generalized belief propagation. In: NIPS. (2000) 689-695
[97] Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Intern Journ Comp Vis 47 (2002) 7-42
[98]沈焕锋.场景变化条件下的影像超分辨率重建技术研究[ D] .武汉大学, 2007.
[99] W Zhou, A C Bovik. A universal image quality in dex [ J] . IEEE Signal Processing Let ters, 2002, 9( 3) : 81-84.
[100] A.J.Patti,Y.Altunbasak.Atifact reduction for set theoretic super resolution image reconstruction with edge adaptive constraints and higher-order interpolants [J].IEEE Transactions on Image Processing,2001,6:1446-1451.
[101] M.K.Ng,N.K.Bose.Analysis of displacement errors in high-resolution image reconstruction with multisensors [J].IEEE Transactions on Circuits and Systems-I:Fundamental Theory and Applications,2002,49(6):806-8 l 3.
[102] Z.Wang,A.C.Bovik,H.R.Sheikh,E.P.Simoncelli.Image quality assessment:from error visibility to structural similarity [J].IEEE Transactions on Image Processing,2004,l 3(4):600-612.
[103] M.K.Kundu, S.K.Pal.Automatic selection of object enhancement operator with quantitative justification based on fuzzy set theoretic measures [J].Pattern recognition letters,1990,1l(12):811-829.
[104]钱曾波,刘静字,肖国超.航天摄影测量[M],北京:解放军出版社,1992.
[105]于秀兰.空间多传感器遥感图像融合分类技术的研究[D].哈尔滨:哈尔滨工业大学博士学位论文,1999.
[106]耿则勋.基于小波变换的遥感影像保持量测精度的压缩技术研究[D].郑州:解放军测绘学院博士学位论文,1996.
[107]熊兴华.遗传算法及其在影像处理与分析中的应用[D].郑州:解放军信息工程大学博士学位论文,2003.
[108] David M. Mount and Sunil Arya, ANN: A Library for Approximate Nearest Neighbor Searching ,version 1.1.1 ,http://www.cs.umd.edu/~mount/ANN/,2006
[109] PARK S C, MOON M K P. Super2resolution image reconstruction: A technical overview [J ]. IEEE Signal ProcessingMagazine, 2003, 20 (3) : 21 - 36.
[2] Harris J L. Diffraction and resolving power [J]. Journal of the Optical Society of America, 54(7): 931-936. 1964.
[3] GOODMAN J W. Introduction to Fourier optics [M]. New York: McGraw-Hill, 1968.
[4] Brown H A. Effect of Truncation on Image Enhancement by Prolate Spheroidal Function [J]. J.O.S.A., 59: 228-229, 1969.
[5] Jain A K. Fundamentals of Digital Image Processing [M]. Prentice Hall, Englewood Cliffs. HJ, 1989.
[6] Wadaka S, Sato T. Superresolution in Incoherent Imaging System [J]. J. O. S. A., 65(3):354-355, 1975.
[7] Tsai R Y, Huang T S. Multipleframe image restoration and registration [C]. Advances in Computer Vision and Image Processing. Greenwich, USA: JAI Press Inc: 317-339. 1984
[8] Kim S P, Bose N K, and Valenzuela H M. Recursive reconstruction of high resolution image from noisy undersampled multiframes [J]. IEEE Trans. Acoust., Speech,Signal Processing, 1990,38(6):1 013-1 027.
[9] Kim S P and Su W Y. Recursive high-resolution reconstruction of blurred multiframe images [J]. IEEE Trans. Image Processing, 1993, 2(10): 534-539.
[10] Bose N K, Kim H C, and Valenzuela H M. Recursive implementation of total least squares algorithm for image reconstruction from noisy, undersampled multiframes [A]. in Proc. IEEE Conf. Acoustics, Speech and Signal Processing, Minneapolis, MN, 1993,5: 269-272.
[11] Rhee S H and Kang M G. Discrete cosine transform based regularized high-resolution image reconstruction algorithm [J]. Opt. Eng., 1999,38(8): 1348-1356.
[12] Tom, B.C., Katsaggelos, A.K.: Reconstruction of a high-resolution image by simultaneous registration, restoration, and interpolation of low-resolution images. In: Proceedings of the IEEE International Conference on Image Processing, pp. 539–542.Washington, DC (1995)
[13] Li, S.Z.: Markov Random Field Modeling in Computer Vision. Springer, New York (1995)
[14] Nguyen, N., Milanfar, P.: A wavelet-based interpolation-restoration method for super-resolution. Circuits Syst. and Signal Process.19, 321–338 (2000)
[15] W. T. Freeman, E. Pasztor, and O. Carmichael. Learning low-level vision. International Journal of Computer Vision, 40(1):25–47, 2000.
[16] Capel, D., Zisserman, A.: Super-resolution enhancement of text image sequences. In: Proceedings of the IEEE International Conference on Pattern Recognition, pp. 600–605. Barcelona, Spain(2000)
[17] Hertzmann, A., Jacobs, C.E., Oliver, N., Curless, B., Salesin, D.H.: Image analogies. In: Proceedings of the SIGGRAPH, pp.327–340. Los Angeles (2001)
[18] Morse, B., Schwartzwald, D.: Image magnification using level-set reconstruction. In: Proc. IEEEInternational Conf. Computer Vision. (2001) 333-341
[19] Liu, C., Shum, H.Y., Zhang, C.S.: Two-step approach to hallucinating faces: global parametric model and local nonparametric model. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition. (2001) 192-198
[20] Baker, S., Kanade, T.: Limits on super-resolution and how to break them. IEEE Trans. Pattern Anal. Mach. Intell. 24, 1167–1183 (2002)
[21] Tipping, M.E., Bishop, C.M. : Bayesian image super-resolution. In: Becker, S., Thrun, S., Obermeyer, K. (eds.) Advances in Neural Information Processing Systems, MIT Press, Cambridge (2002)
[22] W. Freeman, T. Jones, E. Pasztor. Example-based super-resolution. IEEE Computer Graphics and Applications, 22(2): 56-65, 2002.
[23] S.Zhao.H.Hart,S.Peng.Wavelet Domain HMT-based Image Super resolution.1EEE InL Conf.Image processing,2003.2(9):933-936
[24] Pickup, L.C., Roberts, S.J., Zisserman, A. : A sampled texture prior for image super-resolution. In: Thrun, S., Saul, L., Scholkopf, B. (eds.) Advances in Neural Information Processing Systems, MIT Press, Cambridge (2003)
[25] J. Sun, N. N. Zheng, H. Tao, H.Y. Shum. Generic Image hallucination with primal sketch prior. In CVPR, 2003.
[26] Lin, Z., Shum, H.-Y.: Fundamental limits of reconstruction-based super-resolution algorithms under local translation. IEEE Trans. Pattern Anal. Mach. Intell. 26, 83–97 (2004)
[27] Kim, K.I., Franz, M., Scholkopf, B.: Kernel hebbian algorithm for single-frame super-resolution. In: Proceedings of the ECCV Workshop on Statistical Learning in Computer Vision, pp. 135–149. Prague, Czech Republic (2004)
[28] He, H., Kondi, L.P.: Resolution enhancement of video sequences with simultaneous estimation of the regularization parameter. J. Electron. Imaging 13, 586–596 (2004)
[29] Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp.275–282. Washington, D.C. (2004)
[30] Chappalli, M.B., Bose, N.K.: Simultaneous noise filtering and super-resolution with second-generation wavelets. IEEE Signal Process. Lett. 12, 772–775 (2005)
[31] Tanaka,M., Okutomi, M.: Theoretical analysis on reconstructionbased super-resolution for an arbitrary PSF. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 947–954. San Diego, CA (2005)
[32] Pham, T.Q., van Vliet, L.J., Schutte, K.: Influence of signalto-noise ratio and point spread function on limits of superresolution. In: Proceedings of the SPIE Conference on Image Processing: Algorithms and Systems IV, pp. 169–180. San Jose, CA(2005)
[33] Robinson, D., Milanfar, P.: Statistical performance analysis of super-resolution. IEEE Trans. Image Process. 15, 1413–1428(2006)
[34] Woods, N.A., Galatsanos, N.P., Katsaggelos, A.K.: Stochastic methods for joint registration, restoration, and interpolation of multiple undersampled images. IEEE Trans. Image Process.15, 201–213 (2006)
[35] Ei-Khamy, S.E., Hadhoud, M.M., Dessouky, M.I., Salam, B.M., Ei-Samie, F.E.: Regularized super-resolution reconstruction of images using wavelet fusion. Opt. Eng. 44,097001.1–097001.10(2005)
[36] Ei-Khamy, S.E., Hadhoud, M.M., Dessouky, M.I., Salam, B.M., Ei-Samie, F.E.: Wavelet fusion: a tool to break the limits on LMMSE image super-resolution. Int. J. Wavel. Multiresolut. Inf. Process. 4, 105–118 (2006)
[37] Ji, H., Fermuller, C.: Wavelet-based super-resolution reconstruction: theory and algorithm. In: Proceedings of the European Conference on Computer Vision, pp. 295–307. Graz, Austria (2006)
[38] Bose,N.K., Ahuja, N.A.: Super resolution and noise filtering using moving least squares. IEEE Trans. Image Process. 15, 2239–2248 (2006)
[39] Pickup, L.C., Capel, D., Roberts, S.J., Zisserman, A. : Bayesian image super-resolution, continued. In: Scholkopf, B., Platt, J., Hoffman, T. (eds.) Advances in Neural Information Processing Systems, pp. 1089–1096. MIT Press, Cambridge (2006)
[40] Lin, Z.-C., He, J.-F., Tang, X.-O., Tang, C.-K.: Limits of learning-based Super-resolution algorithms. In: Proceedings of the IEEE International Conference on Computer Vision. Rio de Janeiro,Brazil (2007)
[41] Tian, J., Ma, K.-K.: Edge-adaptive super-resolution image reconstruction using a Markov chain Monte Carlo approach. In: Proceedings of the International Conference on Information, Communications and Signal Processing. Singapore (2007)
[42] Suresh, K.V., Rajagopalan, A.N.: Robust and computationally efficient super-resolution algorithm. Opt. Soc. Am. J.AOpt. Image Sci. 24, 984–992 (2007)
[43] M. K. Ng, H. Shen, L. Zhang, E. Lam, "A Total Variation Based Super-Resolution Reconstruction Algorithm for Digital Video," EURASIP Journal on Advances in Signal Processing, vol. 2007, no. Article ID 74585, 2007(SCI).
[44] Shen, H., Zhang, L., Huang, B., Li, P.: A MAP approach for joint motion estimation, segmentation, and super resolution. IEEE Trans. Image Process. 16, 479–490 (2007)
[45] Chakrabarti, A., Rajagopalan, A.N., Chellappa, R.: Super-resolution of face images using kernel PCA-based prior. IEEE Trans. Multimed. 9, 888–892 (2007)
[46] Fattal, R.: Upsampling via imposed edges statistics. ACM Trans. Graph. (Proc. of SIGGRAPH) 26 (2007)
[47] Zibetti, M., Bazan, F., Mayer, J.: Determining the regularization parameters for super-resolution problems. Signal Process. 88, 2890–2901 (2008)
[48] Katsaggelos, A.K., Molina, R.: Special issue on super resolution. Comput. J. 52, 1–167 (2008)
[49] Callico, G., Lopez, S., Sosa, O., Lopez, J.F., Sarmiento, R.: Analysis of fast block matching motion estimation algorithms for video super-resolution systems. IEEE Trans. Consum. Electron. 54, 1430–1438 (2008)
[50] Sun J, Sun J, Xu ZB, et al. Image super-resolution using gradient profile prior [C]. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2008, 2471–2478.J.
[51] Shan, Q., Li, Z., Jia, J., Tang, C.K.: Fast image/video upsampling. ACM Trans. Graph. (Proc. of SIGGRAPH ASIA) 27 (2008) 1{7}
[52] Ji, H., Fermuller, C.: Robust wavelet-based super-resolution reconstruction: theory and algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 31, 649–660 (2009)
[53] Tian, J., Ma, K.-K.: A state-space super-resolution approach for video reconstruction. Signal Image Video Process. 3, 217–240 (2009)
[54] Omer O A,Tanaka T. Region-based weighted-norm approach to video super-resolution with adaptive regularization [C] 2009 IEEE International conference on acoustics,speech,and signal processing. Taipei,Taiwan:[s.n.], 2009:833-836.
[55] Protter, M., Elad, M.: Super resolution with probabilistic motion estimation. IEEE Trans. Image Process. 18, 1899–1904(2009)
[56] Zhang, Y., Zhao, D., Ji, X.,Wang, R., Gao,W.: A spatio-temporal auto regressive model for frame rate up conversion. IEEE Trans. Circuits Syst. Video Technol. 19, 1289–1301 (2009)
[57] Glasner D,Bagon S,Irani M. Super-Resolution from a Single Image [C] IEEE 12th International Conference on computer vision. Kyoto,Japan:[s.n.], 2009:349-456.
[58] H. Takeda, P. Milanfar, M. Protter, M. Elad, Super-Resolution Without Explicit Subpixel Motion Estimation, IEEE transactions on image processing, 18 (9) (2009) pp. 1958-1975
[59] M. Protter, M. Elad, H. Takeda, and P. Milanfar. Generalizing the Non-Local-Means to Super-Resolution Reconstruction. IEEE Transactions on Image Processing, Vol. 18, No. 1, Pages 36-51, January 2009.
[60] Yang, J.,Wright, J., Huang, T.,Ma,Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19, 2861–2873(2010)
[61] Belekos, S., Galatsanos, N.P., Katsaggelos, A.K.: Maximum a posteriori video super-resolution using a new multichannel image prior. IEEE Trans. Image Process. 19, 1451–1464 (2010)
[62] Tian, J., Ma,K.-K.: Stochastic super-resolution image reconstruction. J. Vis. Commun. Image Represent. 21, 232–244 (2010)
[63] Shuaicheng Liu, Michael S. Brown, Seon Joo Kim, Yu-Wing Tai. Colorization for Single Image Super Resolution. In Proceedings of ECCV (6)'2010. pp.323~336
[64] Yu-Wing Tai, Shuaicheng Liu, Michael S B and Stephen Lin.: Super resolution using edge prior and single image detail synthesis. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition. (2010) 1{8}
[65] Agrawal A,Gupta M,Veeraraghavan A,et al. Optimal Coded Sampling for Temporal Super-Resolution [C] 2010 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). San Francisco,CA:[s.n.], 2010:599-606.
[66] Zibettia, M.V.W., Baznb, F.S.V., Mayer, J.: Estimation of the parameters in regularized simultaneous super-resolution. Pattern Recognit. Lett. 32, 69–78 (2011)
[67] H. Zimmer and A. Bruhn and J. Weickert }, Freehand HDR Imaging of Moving Scenes with Simultaneous Resolution Enhancement, Computer Graphics Forum (Proceedings of Eurographics) 30,2,405—414(2011)
[68] O. Shahar, A. Faktor and M. Irani, Space-Time Super-Resolution from a Single Video. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2011.
[69] H. Hou and H. Andrews,“Cubic splines for image interpolation and digital filtering,”IEEE Trans. Acoust., Speech, Signal Process., vol. 26, no. 6, pp. 508–517, Dec. 1978.
[70] M Irani, S Peleg. Super resolution from image sequences [A]. In Proceedings of International Conference on Pattern Recognition. Piscataway, NJ, USA.1990:115-120.
[71] M Irani, S Peleg. Improving resolution by image registration [J]. Graph.Models Image Process,1991, 5(53):231-239.
[72] M Irani, S Peleg. Motion analysis for image enhancement. Resolution, occlusion, and transparency [J]. J Vis. Commun. Image, 1993, 4(4):324-336.
[73] S Mann, R W Picard. Virtual bellows: Constructing high quality stills from Video [C]. IEEE Int. Conf. Image Proc. Austin, TX,1994:13-16.
[74] B. C. Tom, A. K. Katsaggelos. Resolution enhancement of video sequences using motion compensation. In: Proc IEEE Int Conf Image Processing. Lausanne, Switzerland. 1996,1:713-716
[75] B. C. Tom, A. K. Katsaggelos. Resolution enhancement of color video. In: Proc Conf Eur Signal Processing. 1996. 1:145-148
[76] SCHULTZ R R.STEVENSON R L. Improved definition video frame enhancement. 1995
[77] R. C. Hardie, K. J. Barnard, and E. E. Armstrong. Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Transactions on Image Processing, 6(12):1621–1633, December, 1997.
[78] Sean Borman and Robert L. Stevenson. Image Resampling and Constraint Formulation for Multi-Frame Super-Resolution Restoration. 5016:208-219, San Jose, CA, USA, Jan, 2003.
[79] H Stark, P Oskoui. High resolution image recovery from image-plane arrays, using convex projections [J]. J. Opt. Soc. Am. A, 1989, 6:1715-1726。
[80] A J Patti,M I Sezan,A M Tekalp. Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time [J]. IEEE Trans. Image Proc. , 1997, 8(6):1064-1076.
[81] P E Eren, M I Sezan, A M Tekalp. Robust, object-based high-resolution image reconstruction from low-resolution video [J]. IEEE Trans. Image Proc. , 1997, 10(6): 1446-1451.
[82] V. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
[83] N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines. New York: Cambridge Univ. Press, 2000.
[84] M. Palaniswami and A. Shilton,“Adaptive support vector machines for regression,”presented at the 9th Int. Conf. Neural Information Processing, 2002.
[85] Nguyen N,MilanFar P,Golub G.Efficient generalized cross validation with application to parametric image restoration and resolution enhancement [J].IEEE Transaction on Image Processing,2001.10(9):1299—1308
[86] Yu-Wing Tai and Shuaicheng Liu and Michael S. Brown and Stephen Lin. Super Resolution using Edge Prior and Singale Image Detail Synthesis, CVPR, 2010
[87] Zeev Farbman, Raanan Fattal, Dani Lischinski, Richard Szeliski. Edge-Preserving Decompositions for Multi-Scale Tone and Detail Manipulation. ACM Transactions on Graphics 27:3,2008
[88] Freedman, Gilad and Fattal, Raanan. Image and Video Upscaling from Local Self-Examples , ACM Trans. Graph. 28(3): 0730-0301 1-10,2010
[89] Allebach and P. W. Wong,Edge-directed interpolation. In Proc. IEEE Int. Conf. Image Processing, 1996,3:707–710.
[90] Xin Li, Michael T.Orchard.New Edge-Directed Interpolation.IEEE TRANSACTIONS ON IMAGE PROCESSING, OCTOBER 2001,10(10).
[91] Nira Shezaft, Hagit Abramov-Segalt, Ilan Sutskovert and Ran Bar-Sella.Adaptive Low Complexity Algorithm For Image Zooming At Fractional Scaling Ratio.IEEE,2000.
[92]吴良武和欧宗英.保持轮廓清晰光滑的灰度图像放大算法.计算机辅助设计与图像学学报,2002,04,14(4).
[93] Osher and Sethian. Fronts propogating with curvature-dependent speed:Algorithms based on Hamilton-Jacobi formulations , Journal of Computational Physics 79 , 1988,12-49.
[94] Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans Pattern Anal Mach Intell 23 (2001) 1222-1239
[95] Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts. IEEE Trans Pattern Anal Mach Intell 26 (2004) 147-59
[96] Yedidia, J.S., Freeman, W.T., Weiss, Y.: Generalized belief propagation. In: NIPS. (2000) 689-695
[97] Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Intern Journ Comp Vis 47 (2002) 7-42
[98]沈焕锋.场景变化条件下的影像超分辨率重建技术研究[ D] .武汉大学, 2007.
[99] W Zhou, A C Bovik. A universal image quality in dex [ J] . IEEE Signal Processing Let ters, 2002, 9( 3) : 81-84.
[100] A.J.Patti,Y.Altunbasak.Atifact reduction for set theoretic super resolution image reconstruction with edge adaptive constraints and higher-order interpolants [J].IEEE Transactions on Image Processing,2001,6:1446-1451.
[101] M.K.Ng,N.K.Bose.Analysis of displacement errors in high-resolution image reconstruction with multisensors [J].IEEE Transactions on Circuits and Systems-I:Fundamental Theory and Applications,2002,49(6):806-8 l 3.
[102] Z.Wang,A.C.Bovik,H.R.Sheikh,E.P.Simoncelli.Image quality assessment:from error visibility to structural similarity [J].IEEE Transactions on Image Processing,2004,l 3(4):600-612.
[103] M.K.Kundu, S.K.Pal.Automatic selection of object enhancement operator with quantitative justification based on fuzzy set theoretic measures [J].Pattern recognition letters,1990,1l(12):811-829.
[104]钱曾波,刘静字,肖国超.航天摄影测量[M],北京:解放军出版社,1992.
[105]于秀兰.空间多传感器遥感图像融合分类技术的研究[D].哈尔滨:哈尔滨工业大学博士学位论文,1999.
[106]耿则勋.基于小波变换的遥感影像保持量测精度的压缩技术研究[D].郑州:解放军测绘学院博士学位论文,1996.
[107]熊兴华.遗传算法及其在影像处理与分析中的应用[D].郑州:解放军信息工程大学博士学位论文,2003.
[108] David M. Mount and Sunil Arya, ANN: A Library for Approximate Nearest Neighbor Searching ,version 1.1.1 ,http://www.cs.umd.edu/~mount/ANN/,2006
[109] PARK S C, MOON M K P. Super2resolution image reconstruction: A technical overview [J ]. IEEE Signal ProcessingMagazine, 2003, 20 (3) : 21 - 36.