实时无缝全景图像拼接技术研究与实现
|
摘要
近年来,随着计算机技术的快速发展,图像处理应用越来越广泛和深入,对具有较大视域全景图像的需求也越来越迫切。全景图像拼接作为新兴技术,短短几年得到了快速发展,受到研究者越来越多的关注。目前全景图像已经成为计算机仿真、计算机视觉模拟、图像处理和计算机特效以及虚拟现实研究中的热点和关键技术,在地质勘测、军事侦查、医学微创手术、航空航天以及视频会议等多个领域发挥着重要作用。
图像拼接是指把多个较小视域的图像拼接成一个较大视域的图像。本文在研究以往拼接技术的基础上,针对当前图像拼接算法运行效率低的现状,采用基于特征点的图像匹配方法和基于双线性插值的加权平均融合方法,通过改进角点检测以及图像特征点匹配策略,设计了新的图像拼接算法。实验表明,该算法能够显著提高图像拼接的速度。本文算法首先对采集到的图像校正去噪尽可能的去除图像噪声；然后改进角点检测过程,在检测角点前先检测图像边缘,仅对处于边缘点的像素使用Harris算法进行角点检测；在特征点匹配时,把待拼接图像按照顺序进行分组,每组以中间图像为基准,同时和两边的图像进行特征点匹配,这样每次都够完成三幅图像匹配,并且每个图像仅被加载和读取一次；最后使用基于双线性插值的加权平均融合法进行图像融合,降低了图像融合的计算量,并且消除了拼接后图像间的缝隙。实验表明该方法和传统的基于特征匹配的图像拼接算法相比,拼接速度快,具有较好的实时性能。
In recent years, with the rapid development of computer technology, image processing applications increasingly extensive and in-depth,as the same time, on the horizon with a large panoramic image of the demand for more becomes more urgent.Panoramic image stitching technology as a rise of emerging technologies will attract more attentions of the researchers. Just a few years it has been rapid development, and become a key technologie in computer simulation, computer vision, simulation, image processing and computer special effects and virtual reality Panoramic image mosaic image processing technology must be as the future of one of the main research site.
Panoramic image mosaic uses multiple images with smaller view to blend into a larger image with much wide view. Based on previous splicing algorithm and technique, this paper proposes a new panoramic image mosaic algorithm.It assembles multiple images with smaller view into a panoramic image which has larger field of vision, high resolution, smooth edge, and seamless.Before image registering, it removes the collected image correction and denoises image noise as much as possible, Then this algorithm detects cornors in images using Harris algorithm base on detecting image edge through Sobel algorithm.When macthing the images, mosaic images are grouped according to their orders,and uses the meddle image as the reference.In this way it can complete three images match each time, and each image is only load and read for one time. Finally, based on bilinear interpolation and using the weighted average fusion, the algorithm of this paper completes image blending.It reduces the amount of computation, and eliminats the gap between the images in stitched image.It shows that this method has more fast speed and better real-time performance comparing with traditional feature-based matching algorithm for image stitching.
图像拼接是指把多个较小视域的图像拼接成一个较大视域的图像。本文在研究以往拼接技术的基础上,针对当前图像拼接算法运行效率低的现状,采用基于特征点的图像匹配方法和基于双线性插值的加权平均融合方法,通过改进角点检测以及图像特征点匹配策略,设计了新的图像拼接算法。实验表明,该算法能够显著提高图像拼接的速度。本文算法首先对采集到的图像校正去噪尽可能的去除图像噪声；然后改进角点检测过程,在检测角点前先检测图像边缘,仅对处于边缘点的像素使用Harris算法进行角点检测；在特征点匹配时,把待拼接图像按照顺序进行分组,每组以中间图像为基准,同时和两边的图像进行特征点匹配,这样每次都够完成三幅图像匹配,并且每个图像仅被加载和读取一次；最后使用基于双线性插值的加权平均融合法进行图像融合,降低了图像融合的计算量,并且消除了拼接后图像间的缝隙。实验表明该方法和传统的基于特征匹配的图像拼接算法相比,拼接速度快,具有较好的实时性能。
In recent years, with the rapid development of computer technology, image processing applications increasingly extensive and in-depth,as the same time, on the horizon with a large panoramic image of the demand for more becomes more urgent.Panoramic image stitching technology as a rise of emerging technologies will attract more attentions of the researchers. Just a few years it has been rapid development, and become a key technologie in computer simulation, computer vision, simulation, image processing and computer special effects and virtual reality Panoramic image mosaic image processing technology must be as the future of one of the main research site.
Panoramic image mosaic uses multiple images with smaller view to blend into a larger image with much wide view. Based on previous splicing algorithm and technique, this paper proposes a new panoramic image mosaic algorithm.It assembles multiple images with smaller view into a panoramic image which has larger field of vision, high resolution, smooth edge, and seamless.Before image registering, it removes the collected image correction and denoises image noise as much as possible, Then this algorithm detects cornors in images using Harris algorithm base on detecting image edge through Sobel algorithm.When macthing the images, mosaic images are grouped according to their orders,and uses the meddle image as the reference.In this way it can complete three images match each time, and each image is only load and read for one time. Finally, based on bilinear interpolation and using the weighted average fusion, the algorithm of this paper completes image blending.It reduces the amount of computation, and eliminats the gap between the images in stitched image.It shows that this method has more fast speed and better real-time performance comparing with traditional feature-based matching algorithm for image stitching.
引文
[1]王伟,陆佩忠.数字图像拼接技术[M].北京：清华大学出版社,2006.1-14.
[2]MeI Slater, Anthony Steed, Rothbaum,Yiorgos Chrysanthou计算机图形学与虚拟环境[M].北京：机械工业出版,2004.17-35.
[3]Francis S Hill.Jr.计算机图形学[M].北京：清华大学出版社,2009.4-9.
[4]Rafaael C.Gonzalez数字图像处理[M].北京：电子工业出版社,2009.16-23.
[5]Grigore Burdea,Philippe Coiffet, Staib LH. Virtual reality technology[M]北京：电子工业出版社,2003.22-25.
[6]Rothbaum, David Nguyen, Yokahama.An Image-Based Approach to Virtual Environment Navigation Computer Graphics,2007.29-38.
[7]Reliski, H.-Y. Shum.Creating Full View Panoramic Mosaics and Environment Maps. Computer Graphics.2009.251-258.
[8]李小娟等ENVI遥感影像处理教程[M].北京：中国环境科学出版社,2007.4-12.
[9]中蔚,曾文琪.虚拟现实技术[M].北京：清华大学出版社,2009.12-57.
[10]Staib LH,et al.Intermodality 3D medical iamge registration with global search.In IEEE workshop on Biomedical Image Analysis,2004,22-234.
[11]Maurer C R,Fitzatrick J M,Maciunas R J, et al.A review of medical image registration. Interactive image-guided neurosurgery,American Association of neurological surgeons.2003.334-379.
[12]Xu Jianbin, An efficient rotation-invariance remote image matching algorithm based on feature points matching. Geoscience and Remote Sensing Symposium, IEEE International,2005.647-649.
[13]赵辉.基于点特征的图像配准算法研究：[硕士学位论文].山东：山东大学,2007.
[14]林璐.图像配准与拼接技术研究：[硕士学位论文].吉林：长春理工大学,2010.
[15]王建华.基于仿射变换的图像序列拼接方法[J].计算机科学,2003,11(4)：34-36
[16]倪维平,边辉,严卫东.基于兴趣点检测和仿射变换模型的序列图像拼接技术的研究与实现[J].计算机科学,2009,39(3)：14-16.
[17]Rafael C. Gonzalez, Richard E, Woods, MultNiew..Digital Image Processing [M]. PublishingHouse of Electronics Industry,2005,14(05):9-10.
[18]Rothbaum B.O,Hodges, L.F. The use of Virtual Reality Exposure in the Treatment of Anxiety Disorders[J]. Behavior Modification,2003,23(4):507-525.
[19]He Qiang,Tian YingLi,Ma SongDe.A Mosaic Technigue Based on Image Registration[J].High Technology Letters,2008,4(11):112-113.
[20]王兆军,田裕鹏.基于Harris LM全局运动估计的电子稳像算法研究[J].应用科技,2010,2(01)：31-34.
[21]Maurer C R,Fitzatrick J M,Maciunas R J, et al.A review of medical image registration. Interactive image-guided neurosurgery,American Association of neurological surgeons.2003.334-379.
[22]B. K. Ghaffary, A. Sawchuk. A survey of new techniques for Ⅰ mageregistration and mapping, Proceedings of the SPIE:Applications of Digital Image Processing, Eng,2003.222-239.
[23]A. Rosenfeld. Computer Vision:A Source of Models for Biological Visual Process. IEEE Trans on Biomed Eng,2009,36(1):83-94.
[24]T. Poggio and V. Torre. Ⅲ-posed problems and regularization analysis in early vision. Artificial Intelligence Lab. Memo,No.773, Massachusetts Institute of Technology,2004.56-68.
[25]Jungpil Shin, Yu Tang. Deghosting for Image Stitching with Automatic Content-Awareness[J]. Pattern Recognition 2010,23(26):26-27.
[26]Lavallee S, Taylor R H, Dupont B. Registration for computer-integrated surgery:methodology,state of the art.Computer-integrated surgery[N].Technology and clinicalapplications.2006.77-97.
[27]Paul Bao,Dan Xu.Complex wavelet-based image mosaic using edge-preserving visual perception modeling[J].The Computer & Graphics,2007,8(02):12-14.
[28]Adhemar Bultheel.Wavelets with applications in signal and image processing. 2003,4(11):14-17.
[29]黄影.基于正交匹配跟踪以及K-SVD的图像融合技术：[硕士学位论文].北京：华北电力大学,2010.
[30]高健,黄心汉,彭刚,王敏,吴祖玉.基于Harris角点和高斯差分的特征点提取算法[J].模式识别与人工智能,2008,4(02)：15-17.
[31]于昕梅,朱林.边缘检测算子及其在火焰图像中的应用[J].计算机工程,2009(4)：35-36
[32]周德龙,潘泉.图像模糊边缘检测的改进算法[N].中国图像图形学报.2001,6(04)：353-358.
[33]吴炯,张秀彬,张峰等.数字图像中边缘算法的实验研究[J].微计算机信息(测控自动化).2004,20(5)：106-107.
[34]王忠华,汪胜前,邓承志等.基于层间相关性的小波边缘检测.江西师范大学学 报(自然科学版),2008,28(5)：379-381.
[35]温兆麟,陈新,郑德涛.用快速归一化互相关进行缺陷检测[N].广州航海高等专科学校学报,2006,10(02)：145-146.
[36]高光珠,李忠武,余珊,何智勇.归一化互相关系数在图像序列目标检测中的应用[J].计算机工程与科学,2005,27(03)：46-47.
[37]周剑军,欧阳宁,张彤,莫建文.基于RANSAC的图像拼接方法[J].计算机工程与设计,2009,8(01)：23-24.
[38]顾费勇.基于图像自适应图像拼接算法研究与实现：[硕士学位论文].浙江：浙江大学,2007.
[39]张洪,黄劲东,范文雷.最小二乘法及其在参数古籍中的应用[N].自动化学报,2010,3(04)：12-15.
[40]David Nguyen,John Canny.MultNiew.Spatially Faithful Group Video Conterenclng.2005,11 (07):34-36.
[41]V Torre. T. Poggio. On Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence.2003,10(04):147-162.
[42]P. Saint-Marc, J. S. Chen, G. Medioni. Adaptive smoothing:a general tool for early vision Pattern Analysis and Machine Intelligence. IEEE Transactions on Vol.13 Issue,2007.514-529.
[43]Andreas Simon,John Canny.The ConeTA Ponommic Display System for Virtual Environments,2008.3-7.
[44]Rosenfeld. Computer Vision Source of Models for Biological Visual Process. IEEE Trans on Biomed Eng,2009,36(1):83-94.
[45]Reliski, H.-Y. Shum.Creating Full View Panoramic Mosaics and Environment Maps. Computer Graphics,2007.251-258.
[46]B. Buxton. Early Image Processing Structural Techniques Motivated by Human Visual Response. University of Surrey,2004.8-13.
[47]L. G. Brown. A survey of image registration techniques[J]. ACM Computer Surveys,2002,2(03):3-4.
[48]陈传波,陆枫.计算机图形学基础[M].北京：电子工业出版社,2004.114-116.
[49]Maintz JBA,et al. A survey of medical image registration medical. image analysis,2004,4(01):1-3.
[50]Maurer C R,Fitzatrick J M,Maciunas R J, et al.A review of medical image registration. Interactive image-guided neurosurgery.American Association of neurological surgeons,2003.334-379.
[51]刘恩锴.基于特征的图像拼接技术研究：[硕士学位论文].吉林：吉林大学,2009.
[52]赵万金.图像自动拼接技术研究与应用[N].苏州大学学报,2008,2(02)：14-16.
[53]王宇,王涌天,刘越.基于SIFT和小波变换的图像拼接算法[N].北京理工大学学报,2009,13(06)：24-25.
[54]周定富,何明一,杨青.一种基于特征点的稳健无缝图像拼接算法[J].测控技术,2009,7(04)：11-13.
[55]J. J. Clark. Authenticating edges produced by zero-crossing algorithm[J]. IEEE Trans on PAMI,2008,11(1):43-57.
[56]He Qiang,Tian YingLi,Ma SongDe.A Mosaic Technigue Based on Image Registration[N].High Technology Letters,2008,4(11):7-10.
[57]葛云,舒华忠,罗立民.基于Legendre正交矩的配准方法及其在二值图像配准中的应用[N].电子学报,2007,1(02)：3-5.
[58]彭士纯.基于多尺度分析的医学图像配准与融合方法研究：[硕士学位论文].武汉：华中科技大学.2006.
[59]孙晶晶,杨民,刘静华,吴文晋.基于正弦图的计算机断层图像配准[N].北京航空航天大学学报,2011,8(02)：12-15.
[60]王慧燕.图像边缘检测和图像匹配研究及应用：[博士学位论文].浙江：浙江大学,2003.
[2]MeI Slater, Anthony Steed, Rothbaum,Yiorgos Chrysanthou计算机图形学与虚拟环境[M].北京：机械工业出版,2004.17-35.
[3]Francis S Hill.Jr.计算机图形学[M].北京：清华大学出版社,2009.4-9.
[4]Rafaael C.Gonzalez数字图像处理[M].北京：电子工业出版社,2009.16-23.
[5]Grigore Burdea,Philippe Coiffet, Staib LH. Virtual reality technology[M]北京：电子工业出版社,2003.22-25.
[6]Rothbaum, David Nguyen, Yokahama.An Image-Based Approach to Virtual Environment Navigation Computer Graphics,2007.29-38.
[7]Reliski, H.-Y. Shum.Creating Full View Panoramic Mosaics and Environment Maps. Computer Graphics.2009.251-258.
[8]李小娟等ENVI遥感影像处理教程[M].北京：中国环境科学出版社,2007.4-12.
[9]中蔚,曾文琪.虚拟现实技术[M].北京：清华大学出版社,2009.12-57.
[10]Staib LH,et al.Intermodality 3D medical iamge registration with global search.In IEEE workshop on Biomedical Image Analysis,2004,22-234.
[11]Maurer C R,Fitzatrick J M,Maciunas R J, et al.A review of medical image registration. Interactive image-guided neurosurgery,American Association of neurological surgeons.2003.334-379.
[12]Xu Jianbin, An efficient rotation-invariance remote image matching algorithm based on feature points matching. Geoscience and Remote Sensing Symposium, IEEE International,2005.647-649.
[13]赵辉.基于点特征的图像配准算法研究：[硕士学位论文].山东：山东大学,2007.
[14]林璐.图像配准与拼接技术研究：[硕士学位论文].吉林：长春理工大学,2010.
[15]王建华.基于仿射变换的图像序列拼接方法[J].计算机科学,2003,11(4)：34-36
[16]倪维平,边辉,严卫东.基于兴趣点检测和仿射变换模型的序列图像拼接技术的研究与实现[J].计算机科学,2009,39(3)：14-16.
[17]Rafael C. Gonzalez, Richard E, Woods, MultNiew..Digital Image Processing [M]. PublishingHouse of Electronics Industry,2005,14(05):9-10.
[18]Rothbaum B.O,Hodges, L.F. The use of Virtual Reality Exposure in the Treatment of Anxiety Disorders[J]. Behavior Modification,2003,23(4):507-525.
[19]He Qiang,Tian YingLi,Ma SongDe.A Mosaic Technigue Based on Image Registration[J].High Technology Letters,2008,4(11):112-113.
[20]王兆军,田裕鹏.基于Harris LM全局运动估计的电子稳像算法研究[J].应用科技,2010,2(01)：31-34.
[21]Maurer C R,Fitzatrick J M,Maciunas R J, et al.A review of medical image registration. Interactive image-guided neurosurgery,American Association of neurological surgeons.2003.334-379.
[22]B. K. Ghaffary, A. Sawchuk. A survey of new techniques for Ⅰ mageregistration and mapping, Proceedings of the SPIE:Applications of Digital Image Processing, Eng,2003.222-239.
[23]A. Rosenfeld. Computer Vision:A Source of Models for Biological Visual Process. IEEE Trans on Biomed Eng,2009,36(1):83-94.
[24]T. Poggio and V. Torre. Ⅲ-posed problems and regularization analysis in early vision. Artificial Intelligence Lab. Memo,No.773, Massachusetts Institute of Technology,2004.56-68.
[25]Jungpil Shin, Yu Tang. Deghosting for Image Stitching with Automatic Content-Awareness[J]. Pattern Recognition 2010,23(26):26-27.
[26]Lavallee S, Taylor R H, Dupont B. Registration for computer-integrated surgery:methodology,state of the art.Computer-integrated surgery[N].Technology and clinicalapplications.2006.77-97.
[27]Paul Bao,Dan Xu.Complex wavelet-based image mosaic using edge-preserving visual perception modeling[J].The Computer & Graphics,2007,8(02):12-14.
[28]Adhemar Bultheel.Wavelets with applications in signal and image processing. 2003,4(11):14-17.
[29]黄影.基于正交匹配跟踪以及K-SVD的图像融合技术：[硕士学位论文].北京：华北电力大学,2010.
[30]高健,黄心汉,彭刚,王敏,吴祖玉.基于Harris角点和高斯差分的特征点提取算法[J].模式识别与人工智能,2008,4(02)：15-17.
[31]于昕梅,朱林.边缘检测算子及其在火焰图像中的应用[J].计算机工程,2009(4)：35-36
[32]周德龙,潘泉.图像模糊边缘检测的改进算法[N].中国图像图形学报.2001,6(04)：353-358.
[33]吴炯,张秀彬,张峰等.数字图像中边缘算法的实验研究[J].微计算机信息(测控自动化).2004,20(5)：106-107.
[34]王忠华,汪胜前,邓承志等.基于层间相关性的小波边缘检测.江西师范大学学 报(自然科学版),2008,28(5)：379-381.
[35]温兆麟,陈新,郑德涛.用快速归一化互相关进行缺陷检测[N].广州航海高等专科学校学报,2006,10(02)：145-146.
[36]高光珠,李忠武,余珊,何智勇.归一化互相关系数在图像序列目标检测中的应用[J].计算机工程与科学,2005,27(03)：46-47.
[37]周剑军,欧阳宁,张彤,莫建文.基于RANSAC的图像拼接方法[J].计算机工程与设计,2009,8(01)：23-24.
[38]顾费勇.基于图像自适应图像拼接算法研究与实现：[硕士学位论文].浙江：浙江大学,2007.
[39]张洪,黄劲东,范文雷.最小二乘法及其在参数古籍中的应用[N].自动化学报,2010,3(04)：12-15.
[40]David Nguyen,John Canny.MultNiew.Spatially Faithful Group Video Conterenclng.2005,11 (07):34-36.
[41]V Torre. T. Poggio. On Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence.2003,10(04):147-162.
[42]P. Saint-Marc, J. S. Chen, G. Medioni. Adaptive smoothing:a general tool for early vision Pattern Analysis and Machine Intelligence. IEEE Transactions on Vol.13 Issue,2007.514-529.
[43]Andreas Simon,John Canny.The ConeTA Ponommic Display System for Virtual Environments,2008.3-7.
[44]Rosenfeld. Computer Vision Source of Models for Biological Visual Process. IEEE Trans on Biomed Eng,2009,36(1):83-94.
[45]Reliski, H.-Y. Shum.Creating Full View Panoramic Mosaics and Environment Maps. Computer Graphics,2007.251-258.
[46]B. Buxton. Early Image Processing Structural Techniques Motivated by Human Visual Response. University of Surrey,2004.8-13.
[47]L. G. Brown. A survey of image registration techniques[J]. ACM Computer Surveys,2002,2(03):3-4.
[48]陈传波,陆枫.计算机图形学基础[M].北京：电子工业出版社,2004.114-116.
[49]Maintz JBA,et al. A survey of medical image registration medical. image analysis,2004,4(01):1-3.
[50]Maurer C R,Fitzatrick J M,Maciunas R J, et al.A review of medical image registration. Interactive image-guided neurosurgery.American Association of neurological surgeons,2003.334-379.
[51]刘恩锴.基于特征的图像拼接技术研究：[硕士学位论文].吉林：吉林大学,2009.
[52]赵万金.图像自动拼接技术研究与应用[N].苏州大学学报,2008,2(02)：14-16.
[53]王宇,王涌天,刘越.基于SIFT和小波变换的图像拼接算法[N].北京理工大学学报,2009,13(06)：24-25.
[54]周定富,何明一,杨青.一种基于特征点的稳健无缝图像拼接算法[J].测控技术,2009,7(04)：11-13.
[55]J. J. Clark. Authenticating edges produced by zero-crossing algorithm[J]. IEEE Trans on PAMI,2008,11(1):43-57.
[56]He Qiang,Tian YingLi,Ma SongDe.A Mosaic Technigue Based on Image Registration[N].High Technology Letters,2008,4(11):7-10.
[57]葛云,舒华忠,罗立民.基于Legendre正交矩的配准方法及其在二值图像配准中的应用[N].电子学报,2007,1(02)：3-5.
[58]彭士纯.基于多尺度分析的医学图像配准与融合方法研究：[硕士学位论文].武汉：华中科技大学.2006.
[59]孙晶晶,杨民,刘静华,吴文晋.基于正弦图的计算机断层图像配准[N].北京航空航天大学学报,2011,8(02)：12-15.
[60]王慧燕.图像边缘检测和图像匹配研究及应用：[博士学位论文].浙江：浙江大学,2003.