基于改进特征点匹配的电子稳像算法研究
|
摘要
电子稳像是一种新型的视频图像序列稳定技术,它利用计算机数字图像处理和电子技术相结合的方法来实现图像序列的稳定。由于具有稳像精度高、体积小、功耗低、成本低以及易于操作等优点,电子稳像技术已被广泛用于摄像机抖动的检测和消除,视频压缩,运动目标的检测和跟踪,以及图像拼接等领域。本课题在苏州大学现代光学技术研究所“天巡一号小卫星CCD相机研制”的项目资助下,对电子稳像技术进行了深入研究,有效地实现了视频图像的实时稳定。
论文阐述了电子稳像的基本原理、基本方法和处理过程,并讨论了其实现的关键技术。介绍了电子稳像中几种常见的典型算法,同时分析了各算法的优缺点,并针对本课题的要求,分析了实时电子稳像的特点,明确实时稳像算法的要求,最终确定采用基于特征点匹配的电子稳像算法。
论文在研究传统的特征点匹配电子稳像算法的基础上,提出了一种改进的特征点匹配算法的电子稳像技术。此方法通过合理的区域划分,进行图像的特征点提取;然后根据特征点集内具有稳定相对位置的结构特征,采用距离不变准则,对特征点的有效性进行验证,保证各点的局部运动具有良好的全局一致性,建立二维运动模型进行全局运动估计;最后利用均值滤波的方法来进行运动补偿,从而实现图像序列的稳定。在进行特征点提取的过程中,课题研究了几种典型的特征点提取算法,并进行实验室仿真和分析,根据本课题要求选取基于Log操作数边缘点提取方法,并采用图像多分辨率分层技术来提高算法的计算速度。
此外,论文介绍了电子稳像算法的评价方法。同时还建立电子稳像的软件仿真平台,对基于改进特征点的电子稳像算法进行了稳像处理研究,并利用评价方法对稳像效果进行分析,实验结果表明此算法具有良好的稳像效果。
Electronic image stabilization is a new technology to stabilize image sequence, which is combined of computer digital image processing and electronic technology to achieve the stability in the image sequence. It has a lot of advantages, such as high image stabilizing precision, compact size, low power consumption, reasonable price and easy to operate. Therefore it has been widely used in the detection and elimination of video's dithering, video compression, the moving target detection and tracking, image mosaic, etc. This thesis, supported by the item of the design of CCD camera on the 1st Tianxun Moonlet, has carried out an intensive study of the electronic image stabilization technique, and achieved the effective stabilization of the video image sequence.
This thesis expatiates the basic principles, methods and process of the electronic image stabilization, and discusses the key technique of realization. Basing on the study of several typical algorithms for video stabilization and real-time electronic image stabilization algorithm's requirements, feature points matching algorithm is applied to electronic image stabilization.
Basing on the research of the traditional feature points matching algorithm, this thesis presents an improved feature points matching algorithm in electronic image stabilization. Through dividing image into many blocks in reason, the suitable feature points can be selected, which has a stable structure and the relative positions of all points are stable. So the thesis proposes the distance invariant criterion to check the feature matching, which can keep global coherence of all local motions and produce the global motion vector by establishing two-dimensional model of movement. Finally, the average filter is applied to motion compensation to achieve the stability of the image sequence. The thesis also presents several typical feature point extraction algorithms, and has laboratory simulation and analysis. In accordance with the requirements of our item, the method of edge points extraction based on Log operator has been selected, using multi-resolution image layering technology to improve the calculation speed.
In addition, the thesis introduces the evaluation methods of image stabilization. And a software simulated platform has been created to analyze and testify the performance of improved feature tracking algorithm. And the results of experiment show that this method has good performances in image stabilization
论文阐述了电子稳像的基本原理、基本方法和处理过程,并讨论了其实现的关键技术。介绍了电子稳像中几种常见的典型算法,同时分析了各算法的优缺点,并针对本课题的要求,分析了实时电子稳像的特点,明确实时稳像算法的要求,最终确定采用基于特征点匹配的电子稳像算法。
论文在研究传统的特征点匹配电子稳像算法的基础上,提出了一种改进的特征点匹配算法的电子稳像技术。此方法通过合理的区域划分,进行图像的特征点提取;然后根据特征点集内具有稳定相对位置的结构特征,采用距离不变准则,对特征点的有效性进行验证,保证各点的局部运动具有良好的全局一致性,建立二维运动模型进行全局运动估计;最后利用均值滤波的方法来进行运动补偿,从而实现图像序列的稳定。在进行特征点提取的过程中,课题研究了几种典型的特征点提取算法,并进行实验室仿真和分析,根据本课题要求选取基于Log操作数边缘点提取方法,并采用图像多分辨率分层技术来提高算法的计算速度。
此外,论文介绍了电子稳像算法的评价方法。同时还建立电子稳像的软件仿真平台,对基于改进特征点的电子稳像算法进行了稳像处理研究,并利用评价方法对稳像效果进行分析,实验结果表明此算法具有良好的稳像效果。
Electronic image stabilization is a new technology to stabilize image sequence, which is combined of computer digital image processing and electronic technology to achieve the stability in the image sequence. It has a lot of advantages, such as high image stabilizing precision, compact size, low power consumption, reasonable price and easy to operate. Therefore it has been widely used in the detection and elimination of video's dithering, video compression, the moving target detection and tracking, image mosaic, etc. This thesis, supported by the item of the design of CCD camera on the 1st Tianxun Moonlet, has carried out an intensive study of the electronic image stabilization technique, and achieved the effective stabilization of the video image sequence.
This thesis expatiates the basic principles, methods and process of the electronic image stabilization, and discusses the key technique of realization. Basing on the study of several typical algorithms for video stabilization and real-time electronic image stabilization algorithm's requirements, feature points matching algorithm is applied to electronic image stabilization.
Basing on the research of the traditional feature points matching algorithm, this thesis presents an improved feature points matching algorithm in electronic image stabilization. Through dividing image into many blocks in reason, the suitable feature points can be selected, which has a stable structure and the relative positions of all points are stable. So the thesis proposes the distance invariant criterion to check the feature matching, which can keep global coherence of all local motions and produce the global motion vector by establishing two-dimensional model of movement. Finally, the average filter is applied to motion compensation to achieve the stability of the image sequence. The thesis also presents several typical feature point extraction algorithms, and has laboratory simulation and analysis. In accordance with the requirements of our item, the method of edge points extraction based on Log operator has been selected, using multi-resolution image layering technology to improve the calculation speed.
In addition, the thesis introduces the evaluation methods of image stabilization. And a software simulated platform has been created to analyze and testify the performance of improved feature tracking algorithm. And the results of experiment show that this method has good performances in image stabilization
引文
[1]K.Ratakonda.Real-time Digital Video Stabilization for Multi-media Applications[J].IEEE Trans on CAS for Video Tech,1998,8(6):69-72.
[2]沈宏海,刘晶红等.摄像稳定技术[J].光学精密工程,2001,9(2):115-120。
[3]M.Oshima,et al.VHS camcorder with electronic image stabilizer[J].IEEE Trans.Consumer Electronics,Nov.1989,35(4):749-758.
[4]Y.Egusa,et al.An application of fuzzy set theory for an electronic video camera image stabilizer[J].Transactions on Fuzzy Systems,Aug.1995,3(3):351-356.
[5]韩绍坤,赵跃进,刘明奇.电子稳像技术及其发展[[J].光学技术,2001,27(1):71-73。
[6]徐之海,李奇,冯华君.摄像、照相系统中的稳像技术[J].照相机,2002,6:4-5。
[7]程锋,任戈.惯性稳定平台建模及振动传递率分析[J].光电工程,2006,33(4):19-22。
[8]D.N.Eskov,Yu.A.Stepin,VA.Toropin.Optical image stabilization methods and means[J].Sov.J.Opt.Technol,Jan.1982,49(1).
[9]朱强华,李胜勇,姜涛.稳像技术[J].舰船电子对抗,2006,29(5):77-80。
[10]Y.Netser.Image Stabilization[J].Opt.End,1982,11(9).
[11]韩绍坤,赵跃进,刘明奇.电子稳像技术及发展[J].光学技术,2001,27(1):71-73。
[12]董立羽,卜彦龙,戴彬.电子稳像技术发展评述[J].信息技术与信息化,2004,6:17-20。
[13]赵红颖,金宏,熊经武.电子稳像技术概述[J].光学精密工程,2001,9(4):354-35。
[14]钟平.机载电子稳像技术[D].长春光机所钟平博士论文,2003。
[15]Y.CHANG,W.F.HU,M.H.CHENG,et al.Digital Image Translational and Rotational Motion Stabilization Using Optical Flow Technique[J].IEEE Transaction on Consumer Electronics,2002,48(1):108-115.
[16]E.Catmull,A.R.Smith.3-D Transformation of Images in Scanline Order[J].Compute Graphics,1980,14(3):279-285.
[17]R.Kenneth,Castleman.Digital Image Processing[C].Prentice-Hall,1996:95-99.
[18]Carlos.Electronic digital image stabilization design and evaluation with applications[D].University of Maryland.1997.
[19]韩绍坤,张立华,刘明奇.电子稳像技术中的坐标变换及像偏移函数分析[J].光学技术,2002,28(3):278-251。
[20]Q.Zheng,R.Chllappa.A computational vision approach to image registration[J].IEEE Trans.Image Processing,1993(2).
[21]F.Vella,A.Castorina,M.Mancuso,et al.Digital image stabilization by adaptive block motion vectors filtering[J].IEEE Trans on Consumer Electronics,2002,48(3):796-801.
[22]陈志坚,李哲,石磊.一种基于图像特征块匹配的电子稳像算法[J].科学技术与工程,2007,7(11):2512-2515。
[23]朱娟娟,郭宝龙,冯宗哲.一种基于灰度投影算法的电子稳像算法[J].光子学报,2005,33(8):1266-1269。
[24]J.Mahenx,J.Cruikshank,et al.Video rate image stabilization system[J].SPIE,2000,34(14):232-238.
[25]孙辉,张永祥等.高分辨率灰度投影算法及其在电子稳像中的应用[J].光学技术,2006,32(3):378-380。
[26]钟平,于前洋,金光.基于特征点匹配技术的运动估计及补偿方法[J].光电子激光,2004,15(1):72-77。
[27]J.B.Shi,C.Tomasi.Good Features to Track[J].IEEE Conference on Computer Vision and Pattern Recognition,CVPR,1994:593-600.
[28]罗诗途,张玘等.一种基于特征匹配的实时电子稳像算法[J].国防科技大学学报,2005,27(3):45-48。
[29]Sung-Jea Ko,Sung-Hee Lee,Kyung-Hoon Lee.Digital image stabilizing algorithms based on bit-plane matching[J].IEEE Transactions on Consumer Electronics,1998,44:617-622.
[30]S.W.Jeon,et al.Fast digital image stabilizer based on Gray-coded bit-plane matching[J].IEEE Trans Consumer Electronics,Aug.1999,45(3):598-603.
[31]Y.Chan,S.Y.Kung.Multi-level Pixel Difference Classification Methods[J].IEEE,1995:252-255.
[32]S.Ertiirk.Real-time digital image stabilization using Kalman filters[J].Real-Time Imaging,2002,8:317-328.
[33]Jung Youp Suk,Gun-Woo Lee.New Electronic Digital Image Stabilization Algorithm in Wavelet Transform Domain(M).CIS,2005,PART Ⅱ:911-916.
[34]钟平.一种改进的代表点匹配算法在稳像技术中的应用[J].光学技术,2005,31(5):742-745。
[35]S.Erturk.Digital image stabilization with sub-image phase correlation based global motion estimation[J].IEEE Trans.Consumer Electronics,Nov.2003,49(4):1320-1325.
[36]M.K.Kim,et al.An efficient global motion characterization method for image processing applications[J].IEEE Trans.Consumer Electronics,Nov.1997,43(4):1010-1018.
[37]S.Erturk.Image sequence stabilization:motion vector integration(MVI)versus frame position smoothing(FPS)[J].Proc.of the 2nd International Symposium on Image and Signal Processing and Analysis,2001:266-271.
[38]D.Marr,E.Hildreth.Theory of edge detection[J].Proceedings Royal Society of London,1980,B207(1167):187-217.
[39]Canny;John.A Computational Approach to Edge Detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1986,8(6):679-698.
[40]L.Kitchen,A.Rosenfeld.Gray Level Corner Detection[J].Patter Recognition Letters,1982,1(1):95-102.
[41]M.Smiths,J.M.Brady,Susan.A New Approach to Low Level Image Processing[R].Internal Technical Report,Defense Research Agency,UK:Oxford University,1995.
[42]C.Harris,Stephens M,A Combined Corner and Edge Detector[C].In Proc Alley Conf Vision England Manchester University Press,1988:189-192.
[43]官云兰,张红军,刘向美.点特征提取算法探讨[J].东华理工学院学报,2007,30(1):42-46。
[44]王舒鹏,方莉.利用Moravec操作数提取特征点实现过程分析[J].计算机知识与技术,2006,3:125-126。
[45]W.Forstner,E.Gulch.A fast operator for detection and precise location of distinct points,corners and centers of circular features[C].Proceeding of Intercom-mission Workshop on Fast Processing of Photogram metric Data.Interlaken:Switzerland,1987.
[46]Uomori Kenya,Moritura Atsushi,Hirofumi Ishii.Automatic image stabilization system by full-digital signal processing[C].IEEE Int.Conf.Consum.Electron(ICCE).Rosemont,1990.
[47]朱娟娟,郭宝龙.电子稳像的特征点跟踪算法[J].光学学报,2006,26(4):516-521。
[48]Zhong Zhang,R.S.A.Blum.Categorization of multi scale decomposition based image fusion schemes with a performance study for a digital camera application[J].Proceedings of the IEEE,1999,87(8):1315-1326.
[2]沈宏海,刘晶红等.摄像稳定技术[J].光学精密工程,2001,9(2):115-120。
[3]M.Oshima,et al.VHS camcorder with electronic image stabilizer[J].IEEE Trans.Consumer Electronics,Nov.1989,35(4):749-758.
[4]Y.Egusa,et al.An application of fuzzy set theory for an electronic video camera image stabilizer[J].Transactions on Fuzzy Systems,Aug.1995,3(3):351-356.
[5]韩绍坤,赵跃进,刘明奇.电子稳像技术及其发展[[J].光学技术,2001,27(1):71-73。
[6]徐之海,李奇,冯华君.摄像、照相系统中的稳像技术[J].照相机,2002,6:4-5。
[7]程锋,任戈.惯性稳定平台建模及振动传递率分析[J].光电工程,2006,33(4):19-22。
[8]D.N.Eskov,Yu.A.Stepin,VA.Toropin.Optical image stabilization methods and means[J].Sov.J.Opt.Technol,Jan.1982,49(1).
[9]朱强华,李胜勇,姜涛.稳像技术[J].舰船电子对抗,2006,29(5):77-80。
[10]Y.Netser.Image Stabilization[J].Opt.End,1982,11(9).
[11]韩绍坤,赵跃进,刘明奇.电子稳像技术及发展[J].光学技术,2001,27(1):71-73。
[12]董立羽,卜彦龙,戴彬.电子稳像技术发展评述[J].信息技术与信息化,2004,6:17-20。
[13]赵红颖,金宏,熊经武.电子稳像技术概述[J].光学精密工程,2001,9(4):354-35。
[14]钟平.机载电子稳像技术[D].长春光机所钟平博士论文,2003。
[15]Y.CHANG,W.F.HU,M.H.CHENG,et al.Digital Image Translational and Rotational Motion Stabilization Using Optical Flow Technique[J].IEEE Transaction on Consumer Electronics,2002,48(1):108-115.
[16]E.Catmull,A.R.Smith.3-D Transformation of Images in Scanline Order[J].Compute Graphics,1980,14(3):279-285.
[17]R.Kenneth,Castleman.Digital Image Processing[C].Prentice-Hall,1996:95-99.
[18]Carlos.Electronic digital image stabilization design and evaluation with applications[D].University of Maryland.1997.
[19]韩绍坤,张立华,刘明奇.电子稳像技术中的坐标变换及像偏移函数分析[J].光学技术,2002,28(3):278-251。
[20]Q.Zheng,R.Chllappa.A computational vision approach to image registration[J].IEEE Trans.Image Processing,1993(2).
[21]F.Vella,A.Castorina,M.Mancuso,et al.Digital image stabilization by adaptive block motion vectors filtering[J].IEEE Trans on Consumer Electronics,2002,48(3):796-801.
[22]陈志坚,李哲,石磊.一种基于图像特征块匹配的电子稳像算法[J].科学技术与工程,2007,7(11):2512-2515。
[23]朱娟娟,郭宝龙,冯宗哲.一种基于灰度投影算法的电子稳像算法[J].光子学报,2005,33(8):1266-1269。
[24]J.Mahenx,J.Cruikshank,et al.Video rate image stabilization system[J].SPIE,2000,34(14):232-238.
[25]孙辉,张永祥等.高分辨率灰度投影算法及其在电子稳像中的应用[J].光学技术,2006,32(3):378-380。
[26]钟平,于前洋,金光.基于特征点匹配技术的运动估计及补偿方法[J].光电子激光,2004,15(1):72-77。
[27]J.B.Shi,C.Tomasi.Good Features to Track[J].IEEE Conference on Computer Vision and Pattern Recognition,CVPR,1994:593-600.
[28]罗诗途,张玘等.一种基于特征匹配的实时电子稳像算法[J].国防科技大学学报,2005,27(3):45-48。
[29]Sung-Jea Ko,Sung-Hee Lee,Kyung-Hoon Lee.Digital image stabilizing algorithms based on bit-plane matching[J].IEEE Transactions on Consumer Electronics,1998,44:617-622.
[30]S.W.Jeon,et al.Fast digital image stabilizer based on Gray-coded bit-plane matching[J].IEEE Trans Consumer Electronics,Aug.1999,45(3):598-603.
[31]Y.Chan,S.Y.Kung.Multi-level Pixel Difference Classification Methods[J].IEEE,1995:252-255.
[32]S.Ertiirk.Real-time digital image stabilization using Kalman filters[J].Real-Time Imaging,2002,8:317-328.
[33]Jung Youp Suk,Gun-Woo Lee.New Electronic Digital Image Stabilization Algorithm in Wavelet Transform Domain(M).CIS,2005,PART Ⅱ:911-916.
[34]钟平.一种改进的代表点匹配算法在稳像技术中的应用[J].光学技术,2005,31(5):742-745。
[35]S.Erturk.Digital image stabilization with sub-image phase correlation based global motion estimation[J].IEEE Trans.Consumer Electronics,Nov.2003,49(4):1320-1325.
[36]M.K.Kim,et al.An efficient global motion characterization method for image processing applications[J].IEEE Trans.Consumer Electronics,Nov.1997,43(4):1010-1018.
[37]S.Erturk.Image sequence stabilization:motion vector integration(MVI)versus frame position smoothing(FPS)[J].Proc.of the 2nd International Symposium on Image and Signal Processing and Analysis,2001:266-271.
[38]D.Marr,E.Hildreth.Theory of edge detection[J].Proceedings Royal Society of London,1980,B207(1167):187-217.
[39]Canny;John.A Computational Approach to Edge Detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1986,8(6):679-698.
[40]L.Kitchen,A.Rosenfeld.Gray Level Corner Detection[J].Patter Recognition Letters,1982,1(1):95-102.
[41]M.Smiths,J.M.Brady,Susan.A New Approach to Low Level Image Processing[R].Internal Technical Report,Defense Research Agency,UK:Oxford University,1995.
[42]C.Harris,Stephens M,A Combined Corner and Edge Detector[C].In Proc Alley Conf Vision England Manchester University Press,1988:189-192.
[43]官云兰,张红军,刘向美.点特征提取算法探讨[J].东华理工学院学报,2007,30(1):42-46。
[44]王舒鹏,方莉.利用Moravec操作数提取特征点实现过程分析[J].计算机知识与技术,2006,3:125-126。
[45]W.Forstner,E.Gulch.A fast operator for detection and precise location of distinct points,corners and centers of circular features[C].Proceeding of Intercom-mission Workshop on Fast Processing of Photogram metric Data.Interlaken:Switzerland,1987.
[46]Uomori Kenya,Moritura Atsushi,Hirofumi Ishii.Automatic image stabilization system by full-digital signal processing[C].IEEE Int.Conf.Consum.Electron(ICCE).Rosemont,1990.
[47]朱娟娟,郭宝龙.电子稳像的特征点跟踪算法[J].光学学报,2006,26(4):516-521。
[48]Zhong Zhang,R.S.A.Blum.Categorization of multi scale decomposition based image fusion schemes with a performance study for a digital camera application[J].Proceedings of the IEEE,1999,87(8):1315-1326.