基于Dense SIFT特征的无人机影像快速拼接方法
|
摘要
特征匹配是无人机影像拼接过程的关键步骤,针对传统的特征匹配方法在影像拼接过程中获取匹配点少、特征点分布不均匀、匹配耗时长等问题,本文提出一种基于Dense SIFT特征的无人机影像快速拼接算法。首先,利用影像POS信息构建连接矩阵以引导匹配过程;然后在降采样影像上进行影像分块,利用Dense SIFT算子获取初始匹配点,并采用两次NCC方法分别实现降采样影像和原始影像上匹配点的精化;最后,基于共线方程将影像投影至物方面上,完成影像的快速拼接。本文选取2组无人机影像进行拼接实验,将本文算法与SIFT和SURF匹配拼接方法进行对比,结果表明:在影像特征点匹配方面,本文方法获取匹配点数量是SIFT和SURF算法的5倍以上,且匹配点分布更加均匀;在影像拼接结果方面,本文方法不仅能够较快完成影像拼接,而且有效避免了拼接影像中的"重影"现象,保证了较好的拼接质量。
The UAV(Unmanned Aerial Vehicle) photography is a new remote sensing system emerging in recent years. It plays an important role in the rapid emergency response of natural disasters. However, due to the large amount of UAV image data, the traditional method for image matching and mosaic is low accuracy and timeconsuming. Feature matching is one of key steps in UAV image mosaic. Traditional matching algorithms have several problems, including less feature points, feature maldistribution, and time-consuming. To solve these problems, a fast image mosaic algorithm based on Dense SIFT feature matching is proposed. Firstly, the connection matrix is build based on POS(Position and Orientation System) data to conduct the matching process. The UAV images are then down-sampled. Secondly, image segmentation is performed on the downsampled images. Then the Dense SIFT operator is used in overlap area of down-sampled images to obtain the initial matching points which are eliminated through matching by the RANSAC(Random Sample Consensus)algorithm and refined by the NCC algorithm on the original and down-sampled images, respectively. Finally,processed images are projected to the object coordinate system based on collinear equation which is calculated by the bundle adjustment method. By contrast, the SIFT(Scale-Invariant Feature Transform) and SURF(Speeded Up Robust Feature) algorithms and the Pix4Dmapper Photogrammetry software are used to test the quality and efficiency of the Dense SIFT algorithm. Two groups of UAV images mosaic experiment results indicate:(1) The Dense SIFT algorithm can be used to obtain about five times more evenly distributed matching points than the SIFT and SURF algorithm at the same time;(2) The Dense SIFT algorithm can be used to effectively improve the quality of the images mosaic by removing the phenomenon of ghosting;(3) It takes about half the time of Pix4Dmapper software to complete the same image mosaic test using the Dense SIFT algorithm. This indicates that the presented algorithm has a high image mosaic quality and fast processing speed,which can play an important role in the rapid emergency response of natural disasters.
The UAV(Unmanned Aerial Vehicle) photography is a new remote sensing system emerging in recent years. It plays an important role in the rapid emergency response of natural disasters. However, due to the large amount of UAV image data, the traditional method for image matching and mosaic is low accuracy and timeconsuming. Feature matching is one of key steps in UAV image mosaic. Traditional matching algorithms have several problems, including less feature points, feature maldistribution, and time-consuming. To solve these problems, a fast image mosaic algorithm based on Dense SIFT feature matching is proposed. Firstly, the connection matrix is build based on POS(Position and Orientation System) data to conduct the matching process. The UAV images are then down-sampled. Secondly, image segmentation is performed on the downsampled images. Then the Dense SIFT operator is used in overlap area of down-sampled images to obtain the initial matching points which are eliminated through matching by the RANSAC(Random Sample Consensus)algorithm and refined by the NCC algorithm on the original and down-sampled images, respectively. Finally,processed images are projected to the object coordinate system based on collinear equation which is calculated by the bundle adjustment method. By contrast, the SIFT(Scale-Invariant Feature Transform) and SURF(Speeded Up Robust Feature) algorithms and the Pix4Dmapper Photogrammetry software are used to test the quality and efficiency of the Dense SIFT algorithm. Two groups of UAV images mosaic experiment results indicate:(1) The Dense SIFT algorithm can be used to obtain about five times more evenly distributed matching points than the SIFT and SURF algorithm at the same time;(2) The Dense SIFT algorithm can be used to effectively improve the quality of the images mosaic by removing the phenomenon of ghosting;(3) It takes about half the time of Pix4Dmapper software to complete the same image mosaic test using the Dense SIFT algorithm. This indicates that the presented algorithm has a high image mosaic quality and fast processing speed,which can play an important role in the rapid emergency response of natural disasters.
引文
[1]崔红霞,孙杰,林宗坚.无人机遥感设备的自动化控制系统[J].测绘科学,2004(1):47-49,6.[Cui H X,Sun J,Lin ZJ.The auto-control system of remote sensing devices for unmanned air vehicle[J].Science of Surveying and Mapping,2004(1):47-49,6.]
[2]Lowe D G.Distinctive image features from scale-invariant keypoints[C]//International Journal of Computer Vision,2004:91-110.
[3]倪希亮,丁琳,江涛,等.应用不变特征获取均匀控制点的遥感影像配准算法[J].测绘科学,2011,36(2):70-72.[Ni X L,Ding L,Jiang T,et al.A remote sensing image registration algorithm by obtaining uniform control points based on invariant feature[J].Science of Surveying and Mapping,2011,36(2):70-72.]
[4]Moussa A,El-Sheimy N.A fast approach for stitching of aerial images[J].International Archives of Photogrammetry,Remote Sensing and Spatial Information Sciences,2016,XLI-B3:769-774.
[5]刘如飞,卢秀山,刘冰,等.一种改进的无人机航摄影像快速拼接方法[J].测绘通报,2014(2):46-49.[Liu R F,Lu XS,Liu B et al.An improved UAV aerial image quick mosaic method[J].Bulletin of Surveying and Mapping,2014(2):46-49.]
[6]Mehradad S,Satari M,Safdary M,et al.Toward real time UAVS'image mosaicking[J].International Archives of Photogrammetry,Remote Sensing and Spatial Information Sciences,2016,XLI-B1:941-946.
[7]董梅,苏建东,杨举田,等.大区域无人机影像快速无缝拼接方法[J].测绘科学,2014,39(11):129-132.[Dong M,Su J D,Yang J T et al.A fast seamless image mosaic method for UAV images in large areas[J].Science of Surveying and Mapping,2014,39(11):129-132.]
[8]Bay H,Tuytelaars T,Van G L.Surf:Speeded up robust features[C]//European conference on computer vision.Springer,Berlin,Heidelberg,2006:404-417.
[9]胡同喜,牛雪峰,谭洋,等.基于SURF算法的无人机遥感影像拼接技术[J].测绘通报,2015(1):55-58,74.[Hu T X,Niu X F,Tan Y,et al.Unmanned aerial vehicle images mosaic based on SURF algorithm[J].Bulletin of Surveying and Mapping,2015(1):55-58,74.]
[10]王勃,龚志辉,顾广杰,等.一种快速的无人机影像无缝拼接方法[J].测绘科学,2012,37(5):23-26.[Wang B,Gong Z H,Gu G J,et al.A fast method for image registration and seamless mosaicking of UAV[J].Science of Surveying and Mapping,2012,37(5):23-26.]
[11]Rublee E,Rabaud V,Konolige K,et al.ORB:An efficient alternative to SIFT or SURF[C]//Computer Vision(IC-CV),2011 IEEE international conference on.IEEE,2011:2564-2571.
[12]佘建国,徐仁桐,陈宁.基于ORB和改进RANSAC算法的图像拼接技术[J].江苏科技大学学报(自然科学版),2015,29(2):164-169.[She J G,Xu R T,ChenN.Image stitching technology based on ORB and improved RANSAC algorithm[J].Journal of Jiangsu University of Science and Technology(Natural Science Edition),2015,29(2):164-169.]
[13]Mur-Artal R,Tardos J D.ORB-SLAM2:An open-source SLAM system for monocular,stereo,and RGB-D cameras[J].IEEE Transactions on Robotics,2017:1-8.
[14]龚健雅,季顺平.摄影测量与深度学习[J].测绘学报,2018,47(6):693-704.[Gong J Y,Ji S P.Photogrammetry and deep Learning[J].Acta Geodaetica et Cartographica Sinica,2018,47(6):693-704.]
[15]袁修孝,明洋.大重叠度面阵航空数码影像的自动转点[J].武汉大学学报·信息科学版,2009,34(5):505-508.[Yuan X X,Ming Y.Automatic extraction of image tie points for aerial digital frame imagery with large overlap[J].Geomatics and Information Science of Wuhan University,2009,34(5):505-508.]
[16]Li M,Li D,Fan D.A study on automatic UAV image mosaic method for paroxysmal disaster[J].ISPRS-International Archives of the Photogrammetry,Remote Sensing and Spatial Information Sciences,2012,XXXIX-B6:123-128.
[17]Bosch A,Zisserman A,Munoz X.Image classification using random forests and ferns[C]//Computer Vision,2007.ICCV 2007.IEEE 11thInternational Conference on.IEEE,2007:1-8.
[18]杨幸彬,吕京国,张丹璐,等.Dense SIFT与改进最小二乘匹配结合的倾斜航空影像匹配方法[J].测绘通报,2018(10):32-36,70.[Yang X B,Lu J G,Zhang D L,et al.Oblique aerial image matching method based on Dense SIFT and improved least squares matching algorithm[J].Bulletin of Surveying and Mapping,2018(10):32-36,70.]
[19]宫阿都,何孝莹,雷添杰,等.无控制点数据的无人机影像快速处理[J].地球信息科学学报,2010,12(2):2254-2260.[Gong A D,He X Y,Lei T J,et al.Fast image processing method of UAV without control data[J].Journal of GeoInformation Science,2010,12(2):2254-2260.]
[20]Fischler M A,Bolles R C.Random sample consensus:Aparadigm for model fitting with applications to image analysis and automated cartography[J].Communications of the ACM,1981,24(6):381-395.
[21]姜文聪,张继贤,程春泉,等.SIFT与粗差剔除算法相结合的机载SAR影像匹配研究[J].地球信息科学学报,2013,15(3):440-445.[Jiang W C,Zhang J X,Cheng C Q,et al.Matching of airborne SAR images based on a combination of SIFT algorithm with mismatching points eliminated algorithm[J].Journal of Geo-information Science,2013,15(3):440-445.]
[22]Li G.Stereo Matching using normalized cross-correlation in logRGB space[C]//International Conference on Computer Vision in Remote Sensing.IEEE,2013.
[23]Toni Schenk.数字摄影测量学[M].武汉:武汉大学出版社,2009.[Toni Schenk.Digital photogrammetry[M].Wuhan:Wuhan University Press,2009.]
[24]王佩军,徐亚明.摄影测量学[M].武汉:武汉大学出版社,2005.[Wang P J,Xu Y M.Photogrammetry[M].Wuhan:Wuhan University Press,2005.]
[25]赵健,郭忠磊,滕惠忠,等.基于LM方法的单位四元数光束法平差模型解算[J].海洋测绘,2017,37(5):56-59.[Zhao J,Guo Z L,Teng H Z,et al.A solution to unit quaternion bundle adjustment model based on LM method[J].Hydrographic Surveying and Charting,2017,37(5):56-59.]
[26]袁修孝.POS辅助光束法区域网平差[J].测绘学报,2008,37(3):342-348.[Yuan X X.POS-supported bundle block adjustment[J].Acta Geodaetica et Cartographica Sinica,2008,37(3):342-348.]
[27]袁修孝,段梦梦,曹金山.正射影像镶嵌线自动搜索的视差图算法[J].测绘学报,2015,44(8):877-883.[Yuan X X,Duan M M,Cao J S.A seam line detection algorithm for orthophoto mosaicking based on disparity image[J].Acta Geodaetica et Cartographica Sinica,2015,44(8):877-883.]
[28]周清华,潘俊,李德仁.遥感图像镶嵌接缝线自动生成方法综述[J].国土资源遥感,2013,25(2):1-7.[Zhou Q H,Pan J,Li D R.Overview of automatic generation of mosaicking seamlines for remote sensing images[J].Remote Sensing for Land&Resources,2013,25(2):1-7.]
[29]王竞雪,崔昊.局部点、线仿射不变性约束的近景影像直线段匹配[J].地球信息科学学报,2019,21(2):137-146.[Wang J X,Cui H.Line segment matching based on local point-line affine invariance constraints for close-range image[J].Journal of Geo-information Science,2019,21(2):137-146.]
[2]Lowe D G.Distinctive image features from scale-invariant keypoints[C]//International Journal of Computer Vision,2004:91-110.
[3]倪希亮,丁琳,江涛,等.应用不变特征获取均匀控制点的遥感影像配准算法[J].测绘科学,2011,36(2):70-72.[Ni X L,Ding L,Jiang T,et al.A remote sensing image registration algorithm by obtaining uniform control points based on invariant feature[J].Science of Surveying and Mapping,2011,36(2):70-72.]
[4]Moussa A,El-Sheimy N.A fast approach for stitching of aerial images[J].International Archives of Photogrammetry,Remote Sensing and Spatial Information Sciences,2016,XLI-B3:769-774.
[5]刘如飞,卢秀山,刘冰,等.一种改进的无人机航摄影像快速拼接方法[J].测绘通报,2014(2):46-49.[Liu R F,Lu XS,Liu B et al.An improved UAV aerial image quick mosaic method[J].Bulletin of Surveying and Mapping,2014(2):46-49.]
[6]Mehradad S,Satari M,Safdary M,et al.Toward real time UAVS'image mosaicking[J].International Archives of Photogrammetry,Remote Sensing and Spatial Information Sciences,2016,XLI-B1:941-946.
[7]董梅,苏建东,杨举田,等.大区域无人机影像快速无缝拼接方法[J].测绘科学,2014,39(11):129-132.[Dong M,Su J D,Yang J T et al.A fast seamless image mosaic method for UAV images in large areas[J].Science of Surveying and Mapping,2014,39(11):129-132.]
[8]Bay H,Tuytelaars T,Van G L.Surf:Speeded up robust features[C]//European conference on computer vision.Springer,Berlin,Heidelberg,2006:404-417.
[9]胡同喜,牛雪峰,谭洋,等.基于SURF算法的无人机遥感影像拼接技术[J].测绘通报,2015(1):55-58,74.[Hu T X,Niu X F,Tan Y,et al.Unmanned aerial vehicle images mosaic based on SURF algorithm[J].Bulletin of Surveying and Mapping,2015(1):55-58,74.]
[10]王勃,龚志辉,顾广杰,等.一种快速的无人机影像无缝拼接方法[J].测绘科学,2012,37(5):23-26.[Wang B,Gong Z H,Gu G J,et al.A fast method for image registration and seamless mosaicking of UAV[J].Science of Surveying and Mapping,2012,37(5):23-26.]
[11]Rublee E,Rabaud V,Konolige K,et al.ORB:An efficient alternative to SIFT or SURF[C]//Computer Vision(IC-CV),2011 IEEE international conference on.IEEE,2011:2564-2571.
[12]佘建国,徐仁桐,陈宁.基于ORB和改进RANSAC算法的图像拼接技术[J].江苏科技大学学报(自然科学版),2015,29(2):164-169.[She J G,Xu R T,ChenN.Image stitching technology based on ORB and improved RANSAC algorithm[J].Journal of Jiangsu University of Science and Technology(Natural Science Edition),2015,29(2):164-169.]
[13]Mur-Artal R,Tardos J D.ORB-SLAM2:An open-source SLAM system for monocular,stereo,and RGB-D cameras[J].IEEE Transactions on Robotics,2017:1-8.
[14]龚健雅,季顺平.摄影测量与深度学习[J].测绘学报,2018,47(6):693-704.[Gong J Y,Ji S P.Photogrammetry and deep Learning[J].Acta Geodaetica et Cartographica Sinica,2018,47(6):693-704.]
[15]袁修孝,明洋.大重叠度面阵航空数码影像的自动转点[J].武汉大学学报·信息科学版,2009,34(5):505-508.[Yuan X X,Ming Y.Automatic extraction of image tie points for aerial digital frame imagery with large overlap[J].Geomatics and Information Science of Wuhan University,2009,34(5):505-508.]
[16]Li M,Li D,Fan D.A study on automatic UAV image mosaic method for paroxysmal disaster[J].ISPRS-International Archives of the Photogrammetry,Remote Sensing and Spatial Information Sciences,2012,XXXIX-B6:123-128.
[17]Bosch A,Zisserman A,Munoz X.Image classification using random forests and ferns[C]//Computer Vision,2007.ICCV 2007.IEEE 11thInternational Conference on.IEEE,2007:1-8.
[18]杨幸彬,吕京国,张丹璐,等.Dense SIFT与改进最小二乘匹配结合的倾斜航空影像匹配方法[J].测绘通报,2018(10):32-36,70.[Yang X B,Lu J G,Zhang D L,et al.Oblique aerial image matching method based on Dense SIFT and improved least squares matching algorithm[J].Bulletin of Surveying and Mapping,2018(10):32-36,70.]
[19]宫阿都,何孝莹,雷添杰,等.无控制点数据的无人机影像快速处理[J].地球信息科学学报,2010,12(2):2254-2260.[Gong A D,He X Y,Lei T J,et al.Fast image processing method of UAV without control data[J].Journal of GeoInformation Science,2010,12(2):2254-2260.]
[20]Fischler M A,Bolles R C.Random sample consensus:Aparadigm for model fitting with applications to image analysis and automated cartography[J].Communications of the ACM,1981,24(6):381-395.
[21]姜文聪,张继贤,程春泉,等.SIFT与粗差剔除算法相结合的机载SAR影像匹配研究[J].地球信息科学学报,2013,15(3):440-445.[Jiang W C,Zhang J X,Cheng C Q,et al.Matching of airborne SAR images based on a combination of SIFT algorithm with mismatching points eliminated algorithm[J].Journal of Geo-information Science,2013,15(3):440-445.]
[22]Li G.Stereo Matching using normalized cross-correlation in logRGB space[C]//International Conference on Computer Vision in Remote Sensing.IEEE,2013.
[23]Toni Schenk.数字摄影测量学[M].武汉:武汉大学出版社,2009.[Toni Schenk.Digital photogrammetry[M].Wuhan:Wuhan University Press,2009.]
[24]王佩军,徐亚明.摄影测量学[M].武汉:武汉大学出版社,2005.[Wang P J,Xu Y M.Photogrammetry[M].Wuhan:Wuhan University Press,2005.]
[25]赵健,郭忠磊,滕惠忠,等.基于LM方法的单位四元数光束法平差模型解算[J].海洋测绘,2017,37(5):56-59.[Zhao J,Guo Z L,Teng H Z,et al.A solution to unit quaternion bundle adjustment model based on LM method[J].Hydrographic Surveying and Charting,2017,37(5):56-59.]
[26]袁修孝.POS辅助光束法区域网平差[J].测绘学报,2008,37(3):342-348.[Yuan X X.POS-supported bundle block adjustment[J].Acta Geodaetica et Cartographica Sinica,2008,37(3):342-348.]
[27]袁修孝,段梦梦,曹金山.正射影像镶嵌线自动搜索的视差图算法[J].测绘学报,2015,44(8):877-883.[Yuan X X,Duan M M,Cao J S.A seam line detection algorithm for orthophoto mosaicking based on disparity image[J].Acta Geodaetica et Cartographica Sinica,2015,44(8):877-883.]
[28]周清华,潘俊,李德仁.遥感图像镶嵌接缝线自动生成方法综述[J].国土资源遥感,2013,25(2):1-7.[Zhou Q H,Pan J,Li D R.Overview of automatic generation of mosaicking seamlines for remote sensing images[J].Remote Sensing for Land&Resources,2013,25(2):1-7.]
[29]王竞雪,崔昊.局部点、线仿射不变性约束的近景影像直线段匹配[J].地球信息科学学报,2019,21(2):137-146.[Wang J X,Cui H.Line segment matching based on local point-line affine invariance constraints for close-range image[J].Journal of Geo-information Science,2019,21(2):137-146.]