基于特征的图像拼接技术研究
|
摘要
图像拼接是数字图像处理的一个重要研究领域,它是将具有一定重叠的两幅或者多幅图像进行匹配对准,融合形成一幅大视场图像的技术,在医学、工业、航天等诸多领域有着广泛的应用。
由于成像系统特性、拍摄角度和时间的不同,以及噪声干扰和遮挡,使图像拼接变得十分困难。按照图像配准的依据不同,图像拼接分为基于特征、基于灰度信息和基于变换域的图像拼接。基于特征的图像拼接不仅不易受光照、旋转等因素影响,而且特征相对像素数量较少,有利于提高速度,因此成为最受关注的一类拼接方法。在本论文中提出了一种稳健的基于特征点的图像拼接算法。
本文首先介绍了图像拼接的几何基础,并根据平面场景的两图像之间的变换关系,确定了射影变换为本文的变换模型。它将作为本文变换矩阵和融合等计算的基础。
基于特征点的图像拼接主要包括特征点提取、特征点的匹配、变换矩阵的计算和图像融合四步。特征点提取是整个图像拼接过程的第一步,对于最终拼接结果至关重要。本文介绍和实现了Harrris、SUSAN、SIFT三种最为流行的特征点提取算法。由于SIFT算法对平移、尺度缩放、旋转等保持不变性,对光照变换、仿射变换、甚至射影变换也保持一定程度的稳定性,而且SIFT特征描述符具有很强的匹配能力,因此选择了稳健的SIFT算法作为特征点提取算法。在特征点匹配阶段,本文首先对两图的特征点特征描述符建立k-d树,并利用它寻找特征点在另一幅图上的最近邻、次近邻匹配特征点,进而筛选出初始匹配点对。在计算变换矩阵阶段,本文使用PROSAC方法对初始匹配点对进行过滤,得到变换矩阵的初值和内点集合,然后采用L-M算法对变换矩阵参数进行求精。最后,在图像融合时,为了防止灰度拼接缝,本文依据匹配点对的灰度差异,对拼接图像进行匀光处理,使其灰度一致,再根据得到的变换矩阵,进行加权融合。实验结果显示,最终得到的拼接效果良好。
Image stitching is an important research area of image processing. It is a technology that matches a series of images which are overlapped with each other and mixes these images into one image which has big field of view and has been widely used in the medical, industrial, aerospace and other fields.
Because of difference of imaging system, point of view, the time as well as noise interference and obstructions, image stitching becames very difficult. According to the different basis of image registration, image stitching is divided into feature-based, gray-information-based and transformation-domain based. The feature-based image stitching is not only vulnerable to the light, rotation, but also helpful to increasing efficiency because the number of features is quite fewer than image piexel. Therefore, it become one type of most attended methods.This paper presents a robust image stitching algorithm.based on feature points.
At first, the geometric basis of image stitching is introduced, and in accordance with the transformation between the two images of planar scene, projection transformation is choosed as transformation model. So, projection transformation is the basis of computation of transformation matrix and mixing images.
The image stitching based on feature points is mainly composed of four steps: feature points extraction, matching feature points, computing transformation matrix and image mixing. Feature extraction is the first step in the process of image stitching, and is also very crucial. This article introduces and implements Harrris, SUSAN, SIFT three kinds of most popular feature point extraction algorithms. Because the SIFT algorithm is invariant to image translation, scaling, and rotation, and partially invariant to illumination changes and affine or 3D projection, SIFT descriptor has strong ability of matching. So the robust SIFT algorithm. is choosed as feature extraction algorithm. In stage of feature point matching, the k-d tree of SIFT features descriptor is build for two stitching images respectively, then use k-d tree to look for the nearest and the second nearest neighbor feature point in another image for each feature point in one image, and select initial matching point pairs. In the stage of computation of transformation matrix, PROSAC algorithm is used to filter the initial matching point pairs, and get the initial transformation matrix as well as inliers, then the L-M algorithm is applied to obtain higher precision transformation matrix. In the process of image mixing,in order to prevent the gray joints, gray of image is adjusted before mixing image based on the gray differences of inliers., then, according to transformation matrix, weighted mix images. Experiment results show that the quality of final stitching images.is well.
由于成像系统特性、拍摄角度和时间的不同,以及噪声干扰和遮挡,使图像拼接变得十分困难。按照图像配准的依据不同,图像拼接分为基于特征、基于灰度信息和基于变换域的图像拼接。基于特征的图像拼接不仅不易受光照、旋转等因素影响,而且特征相对像素数量较少,有利于提高速度,因此成为最受关注的一类拼接方法。在本论文中提出了一种稳健的基于特征点的图像拼接算法。
本文首先介绍了图像拼接的几何基础,并根据平面场景的两图像之间的变换关系,确定了射影变换为本文的变换模型。它将作为本文变换矩阵和融合等计算的基础。
基于特征点的图像拼接主要包括特征点提取、特征点的匹配、变换矩阵的计算和图像融合四步。特征点提取是整个图像拼接过程的第一步,对于最终拼接结果至关重要。本文介绍和实现了Harrris、SUSAN、SIFT三种最为流行的特征点提取算法。由于SIFT算法对平移、尺度缩放、旋转等保持不变性,对光照变换、仿射变换、甚至射影变换也保持一定程度的稳定性,而且SIFT特征描述符具有很强的匹配能力,因此选择了稳健的SIFT算法作为特征点提取算法。在特征点匹配阶段,本文首先对两图的特征点特征描述符建立k-d树,并利用它寻找特征点在另一幅图上的最近邻、次近邻匹配特征点,进而筛选出初始匹配点对。在计算变换矩阵阶段,本文使用PROSAC方法对初始匹配点对进行过滤,得到变换矩阵的初值和内点集合,然后采用L-M算法对变换矩阵参数进行求精。最后,在图像融合时,为了防止灰度拼接缝,本文依据匹配点对的灰度差异,对拼接图像进行匀光处理,使其灰度一致,再根据得到的变换矩阵,进行加权融合。实验结果显示,最终得到的拼接效果良好。
Image stitching is an important research area of image processing. It is a technology that matches a series of images which are overlapped with each other and mixes these images into one image which has big field of view and has been widely used in the medical, industrial, aerospace and other fields.
Because of difference of imaging system, point of view, the time as well as noise interference and obstructions, image stitching becames very difficult. According to the different basis of image registration, image stitching is divided into feature-based, gray-information-based and transformation-domain based. The feature-based image stitching is not only vulnerable to the light, rotation, but also helpful to increasing efficiency because the number of features is quite fewer than image piexel. Therefore, it become one type of most attended methods.This paper presents a robust image stitching algorithm.based on feature points.
At first, the geometric basis of image stitching is introduced, and in accordance with the transformation between the two images of planar scene, projection transformation is choosed as transformation model. So, projection transformation is the basis of computation of transformation matrix and mixing images.
The image stitching based on feature points is mainly composed of four steps: feature points extraction, matching feature points, computing transformation matrix and image mixing. Feature extraction is the first step in the process of image stitching, and is also very crucial. This article introduces and implements Harrris, SUSAN, SIFT three kinds of most popular feature point extraction algorithms. Because the SIFT algorithm is invariant to image translation, scaling, and rotation, and partially invariant to illumination changes and affine or 3D projection, SIFT descriptor has strong ability of matching. So the robust SIFT algorithm. is choosed as feature extraction algorithm. In stage of feature point matching, the k-d tree of SIFT features descriptor is build for two stitching images respectively, then use k-d tree to look for the nearest and the second nearest neighbor feature point in another image for each feature point in one image, and select initial matching point pairs. In the stage of computation of transformation matrix, PROSAC algorithm is used to filter the initial matching point pairs, and get the initial transformation matrix as well as inliers, then the L-M algorithm is applied to obtain higher precision transformation matrix. In the process of image mixing,in order to prevent the gray joints, gray of image is adjusted before mixing image based on the gray differences of inliers., then, according to transformation matrix, weighted mix images. Experiment results show that the quality of final stitching images.is well.
引文
[1]Sevket Gumustekin.AN INTRODUCTION TO IMAGE MOSAICING.http://www.iyte.edu.tr/eee/sevgum/research/mosaicing99/.2008年3月20.
[2]W.T.Freeman,E.C.Pasztor,and O.T.Carmichael.Learning low-level vision[C].Proc.of International Conference on Computer Vision,1999(2):1182-1189.
[3]李晓娟.图像拼接技术研究[D].西安电子科技大学.硕士学位论文.2007.
[4]陆方杰.基于特征的数字图像拼接技术研究[D].浙江大学.硕士学位论文.2007.
[5]胡社教,葛西旺,陈宗海.基于角点特征的KLT跟踪全景图像拼接算法[J].系统仿真学报.2007,19(8):1742-1744.1753.
[6]李寒,牛纪桢,郭禾.基于特征点的全自动无缝图像拼接方法[J].计算机工程与设计.2007,28(9):2083-2085.
[7]李晓娟.郭宝龙.一种基于兴趣点方向特征的图像拼接方法[J].计算机工程与应用.2007,43(12):26-28.
[8]谭磊,张桦,薛彦斌.一种基于特征点的图像匹配算法[J].天津理工大学学报.2006,22(6):66-69.
[9]仵建宁,郭宝龙,冯宗哲.一种基于兴趣点匹配的图像拼接方法[J].计算机应用.2006,26(3):610-612.
[10]葛西旺.一种精确的图像拼接方法[J].安徽建筑工业学院学报.2007,第15卷第1期:69-72.
[11]徐海云,王普,王广生.全景拼图的实现技术[J]北京工业大学学报.2004,30(4):417-422.
[12]王婉喜,裴仁清.多面阵CCD协同精密检测较大尺寸物体的研究[J].光学仪器.2001,23(4):3-7.
[13]张世阳,王俊杰,胡运发.一种快速全景图像拼接技术[J].计算机应用与软 件.2004,21(3):77-79.
[14]林木华,游志胜,邢霄飞.基于2幂子图像的图像拼接算法[J].中国民航飞行学院学报.2006,17(3):36-38.
[15]徐馄.稳健的图像拼接算法研究[D].南昌大学.硕士学位论文.2005
[16]朱广新.基于特征点匹配的图像拼接及医学应用[D].南京理工大学.硕士学位论文.2007.
[17]汪华琴.基于特征点匹配的图像拼接方法研究[D].华中师范大学.硕士学位论文 2007.
[18]陈利军.图像角点检测和匹配算法得研究[D].西安电子科技大学.硕士学位论文 2005.
[19]李博.图像的不变特征检测与描述研究[D].重庆大学.硕士学位论文.2007
[20]Richard Szeliski.Video mosaics for virtual envirotunents.IEEE Computer GraPhics and Applications,1996.16(2):22-30.
[21]Peleg,S.,Rousso,B.,Rav-Acha,A.,Zomet,A.,2000.Mosaicing on adaptive manifolds.IEEE Trans.on Pattern Analysis and Machine Intelligence,22(10):1144-1154
[22]M.Brown and D.Lowe.Recognising Panoramas[C].Proeeedings of IEEE International Conference on Computer Vision.2003.1218-1225.
[23]K.Mikolajezyk and C.Schmid,.cale and affine[J].invariant interest point detectors.International Journal of Computer Vision,vol.1(60):63-86.
[24]钟力,胡晓峰.重叠图像拼接算法[J].中国图像图学报.1998,3(5):367-370.
[25]李跃,彭宇行,陈福接.一种基于模板匹配的电路照片拼接方法[J].中国图象图形学报.2001,6A(3):301-305.
[26]郭海涛,刘智,张保明.基于遗传算法的快速影像匹配技术的研究[J].测绘学院学报,2001,18:20-22.
[27]张素,王小菊,徐乃平.一种基于小波分解的多图象拼接技术[J].西北工业 大学学报.2001,19(1):23-26.
[28]俞辉,侯在克,何旭莉,吴炎.一种基于轮廓特征的图像拼接算法设计与实现[J].石油大学学报.2003,27(2):114-118.
[29]陈桂兰,解凯.基于自适应流形的图像拼接[J].哈尔滨师范大学自然科学学报.2006,22(1):58-61.
[30]洪春勇,徐琨.一种基于信息熵的灰度失真图像的快速拼接方法[J].南昌大学学报.2006,30(6):605-607,612.
[31]胡社教,葛西旺,陈宗海.基于角点特征的KLT跟踪全景图像拼接算法[J].系统 仿真学报.2007,19(8):1742-1744,1753.
[32]马颂德,张正友.计算机视觉[M].第一版.北京:科学出版社.1996.
[33]R.Szeliski.Image mosaicing for tele-reality applications.In Proc.of IEEE Workshop on Applications of Computer Vision,1994:44-53.
[34]Harris C G,Stephens M J.A combined corner and edge detector[C].In 4th Alvey Vision Conference,Manchester 1988:147-151.
[35]陈白帆,蔡白兴.基于尺度空间理论的Hartis角点检测[J].中南大学学报.2005,36(5):751-754.
[36]李玉山.数字视觉视频技术[M].第一版.西安:西安电子科技大学出版社.2006.
[37]Lowe D G.Object recognition from local scale—invariant features [A].International Conference on Computer Vision[C]。Greece,1999:1150-1157.
[38]Lowe D G.Distinctive Image Features from Scale-Invariant Keypoints[J].International Journal of Computer Vision.2004,60(2):91-110.
[39]Lindeberg T.Scale-space tbeory in computer vision[M].1st ed.,Norwell,MA:Kluwer Academic Publishers,1994.
[40]Lindeberg T.Scale-space theory:A basic tool for analysing structures at different scales[J].Journal Applied Statistics,1994,21(2):223-261.
[41]胡巍,张桂林,陈超阳.一种基于尺度空间理论的高斯平滑方法[J].数据采集与处理.1998,13(3):276-278.
[42]孙剑,徐宗本.计算机视觉中的尺度空间方法[J].工程数学学报.2005,22(6):951-962.
[43]Koenderink J.The Structure of Images[J].Bioiogical Cybernetics.1984,50(5):363-370.
[44]Lindeberg T.Detecting salient blob-like image structures and their scales with a scale-space primal sketch:A method for focus-of-attention.International Journal of Computer Vision,1993,11(3):283-318.
[45]Fabio Remondino.Detectors and Descriptors for Photogrammetric Applications.http://www.photogrammetry.ethz.ch.2008年3月.
[46]张春美,龚志辉,孙雷.改进SIFT特征在图像匹配中的应用[J].计算机工程与应用.2008,44(2).95-97.
[47]求是科技.VISUAL C++图象处理程序[M].第一版.北京:人民邮电出版社.2007.498-500.
[48]L.G.Brown,A survey of image registration techniques.ACM Computing Surveys,1992,24(4):325-376.
[49]Jonathan Klippenstein and Hong Zhang,Quantitative Evaluation of Feature Extractors for Visual SLAM[J].CRV.2007.157-164.
[50]F.Preparata and M.Shamos.Computational Geometry:An Introduction Springer-Verlag,NY,1985.ISBN 3-540-96131-3.
[51]Chum O,Matas J.Matching with PROSAC—progressive sample consensus[A].Proc.CVPR(1).Los Alamitos:IEEE Computer Society,2005,1:20-25.
[52]M.Fischler,R.Bolles.Random sample consensus:A paradigm for model fitting with application to image analysis and automated cartography,Communications of the ACM,1981,24:381-395.
[53]管涛,李利军,段利亚等.基于平面重建与自然特征匹配的虚实注册方法 [J].光电工程.2007,34(9):134-139.
[54]杨华中,汪蕙。数值计算方法与c语言工程函数库[M].北京:科学出版社.1996.
[55]K.Madsen,H.B.Nielsen,O.Tingleff.Methods for non-linear least squares problems.http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/3215/pdf/i mm3215.pdf.2008年4月9日.
[56]葛仕明 程义民 李杰 张玲.基于梯度场的拼接缝消除方法[J].计算机辅助设计与图形学学报.2007,19(2):227-232,239.
[57]潭浩强.C程序设计[M].第一版.北京:清华大学出版社.1991.
[58]钱能.C++程序设计教程[M].第一版.北京:清华大学出版社.1999.
[2]W.T.Freeman,E.C.Pasztor,and O.T.Carmichael.Learning low-level vision[C].Proc.of International Conference on Computer Vision,1999(2):1182-1189.
[3]李晓娟.图像拼接技术研究[D].西安电子科技大学.硕士学位论文.2007.
[4]陆方杰.基于特征的数字图像拼接技术研究[D].浙江大学.硕士学位论文.2007.
[5]胡社教,葛西旺,陈宗海.基于角点特征的KLT跟踪全景图像拼接算法[J].系统仿真学报.2007,19(8):1742-1744.1753.
[6]李寒,牛纪桢,郭禾.基于特征点的全自动无缝图像拼接方法[J].计算机工程与设计.2007,28(9):2083-2085.
[7]李晓娟.郭宝龙.一种基于兴趣点方向特征的图像拼接方法[J].计算机工程与应用.2007,43(12):26-28.
[8]谭磊,张桦,薛彦斌.一种基于特征点的图像匹配算法[J].天津理工大学学报.2006,22(6):66-69.
[9]仵建宁,郭宝龙,冯宗哲.一种基于兴趣点匹配的图像拼接方法[J].计算机应用.2006,26(3):610-612.
[10]葛西旺.一种精确的图像拼接方法[J].安徽建筑工业学院学报.2007,第15卷第1期:69-72.
[11]徐海云,王普,王广生.全景拼图的实现技术[J]北京工业大学学报.2004,30(4):417-422.
[12]王婉喜,裴仁清.多面阵CCD协同精密检测较大尺寸物体的研究[J].光学仪器.2001,23(4):3-7.
[13]张世阳,王俊杰,胡运发.一种快速全景图像拼接技术[J].计算机应用与软 件.2004,21(3):77-79.
[14]林木华,游志胜,邢霄飞.基于2幂子图像的图像拼接算法[J].中国民航飞行学院学报.2006,17(3):36-38.
[15]徐馄.稳健的图像拼接算法研究[D].南昌大学.硕士学位论文.2005
[16]朱广新.基于特征点匹配的图像拼接及医学应用[D].南京理工大学.硕士学位论文.2007.
[17]汪华琴.基于特征点匹配的图像拼接方法研究[D].华中师范大学.硕士学位论文 2007.
[18]陈利军.图像角点检测和匹配算法得研究[D].西安电子科技大学.硕士学位论文 2005.
[19]李博.图像的不变特征检测与描述研究[D].重庆大学.硕士学位论文.2007
[20]Richard Szeliski.Video mosaics for virtual envirotunents.IEEE Computer GraPhics and Applications,1996.16(2):22-30.
[21]Peleg,S.,Rousso,B.,Rav-Acha,A.,Zomet,A.,2000.Mosaicing on adaptive manifolds.IEEE Trans.on Pattern Analysis and Machine Intelligence,22(10):1144-1154
[22]M.Brown and D.Lowe.Recognising Panoramas[C].Proeeedings of IEEE International Conference on Computer Vision.2003.1218-1225.
[23]K.Mikolajezyk and C.Schmid,.cale and affine[J].invariant interest point detectors.International Journal of Computer Vision,vol.1(60):63-86.
[24]钟力,胡晓峰.重叠图像拼接算法[J].中国图像图学报.1998,3(5):367-370.
[25]李跃,彭宇行,陈福接.一种基于模板匹配的电路照片拼接方法[J].中国图象图形学报.2001,6A(3):301-305.
[26]郭海涛,刘智,张保明.基于遗传算法的快速影像匹配技术的研究[J].测绘学院学报,2001,18:20-22.
[27]张素,王小菊,徐乃平.一种基于小波分解的多图象拼接技术[J].西北工业 大学学报.2001,19(1):23-26.
[28]俞辉,侯在克,何旭莉,吴炎.一种基于轮廓特征的图像拼接算法设计与实现[J].石油大学学报.2003,27(2):114-118.
[29]陈桂兰,解凯.基于自适应流形的图像拼接[J].哈尔滨师范大学自然科学学报.2006,22(1):58-61.
[30]洪春勇,徐琨.一种基于信息熵的灰度失真图像的快速拼接方法[J].南昌大学学报.2006,30(6):605-607,612.
[31]胡社教,葛西旺,陈宗海.基于角点特征的KLT跟踪全景图像拼接算法[J].系统 仿真学报.2007,19(8):1742-1744,1753.
[32]马颂德,张正友.计算机视觉[M].第一版.北京:科学出版社.1996.
[33]R.Szeliski.Image mosaicing for tele-reality applications.In Proc.of IEEE Workshop on Applications of Computer Vision,1994:44-53.
[34]Harris C G,Stephens M J.A combined corner and edge detector[C].In 4th Alvey Vision Conference,Manchester 1988:147-151.
[35]陈白帆,蔡白兴.基于尺度空间理论的Hartis角点检测[J].中南大学学报.2005,36(5):751-754.
[36]李玉山.数字视觉视频技术[M].第一版.西安:西安电子科技大学出版社.2006.
[37]Lowe D G.Object recognition from local scale—invariant features [A].International Conference on Computer Vision[C]。Greece,1999:1150-1157.
[38]Lowe D G.Distinctive Image Features from Scale-Invariant Keypoints[J].International Journal of Computer Vision.2004,60(2):91-110.
[39]Lindeberg T.Scale-space tbeory in computer vision[M].1st ed.,Norwell,MA:Kluwer Academic Publishers,1994.
[40]Lindeberg T.Scale-space theory:A basic tool for analysing structures at different scales[J].Journal Applied Statistics,1994,21(2):223-261.
[41]胡巍,张桂林,陈超阳.一种基于尺度空间理论的高斯平滑方法[J].数据采集与处理.1998,13(3):276-278.
[42]孙剑,徐宗本.计算机视觉中的尺度空间方法[J].工程数学学报.2005,22(6):951-962.
[43]Koenderink J.The Structure of Images[J].Bioiogical Cybernetics.1984,50(5):363-370.
[44]Lindeberg T.Detecting salient blob-like image structures and their scales with a scale-space primal sketch:A method for focus-of-attention.International Journal of Computer Vision,1993,11(3):283-318.
[45]Fabio Remondino.Detectors and Descriptors for Photogrammetric Applications.http://www.photogrammetry.ethz.ch.2008年3月.
[46]张春美,龚志辉,孙雷.改进SIFT特征在图像匹配中的应用[J].计算机工程与应用.2008,44(2).95-97.
[47]求是科技.VISUAL C++图象处理程序[M].第一版.北京:人民邮电出版社.2007.498-500.
[48]L.G.Brown,A survey of image registration techniques.ACM Computing Surveys,1992,24(4):325-376.
[49]Jonathan Klippenstein and Hong Zhang,Quantitative Evaluation of Feature Extractors for Visual SLAM[J].CRV.2007.157-164.
[50]F.Preparata and M.Shamos.Computational Geometry:An Introduction Springer-Verlag,NY,1985.ISBN 3-540-96131-3.
[51]Chum O,Matas J.Matching with PROSAC—progressive sample consensus[A].Proc.CVPR(1).Los Alamitos:IEEE Computer Society,2005,1:20-25.
[52]M.Fischler,R.Bolles.Random sample consensus:A paradigm for model fitting with application to image analysis and automated cartography,Communications of the ACM,1981,24:381-395.
[53]管涛,李利军,段利亚等.基于平面重建与自然特征匹配的虚实注册方法 [J].光电工程.2007,34(9):134-139.
[54]杨华中,汪蕙。数值计算方法与c语言工程函数库[M].北京:科学出版社.1996.
[55]K.Madsen,H.B.Nielsen,O.Tingleff.Methods for non-linear least squares problems.http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/3215/pdf/i mm3215.pdf.2008年4月9日.
[56]葛仕明 程义民 李杰 张玲.基于梯度场的拼接缝消除方法[J].计算机辅助设计与图形学学报.2007,19(2):227-232,239.
[57]潭浩强.C程序设计[M].第一版.北京:清华大学出版社.1991.
[58]钱能.C++程序设计教程[M].第一版.北京:清华大学出版社.1999.