艇基低空遥感影像自动拼接关键技术研究
|
摘要
在中国地质调查局《艇基低空高分辨率遥感地质调查关键技术及其应用示范》的项目支持下,基于艇基低空遥感影像,研究实现全景影像图自动拼接过程中的关键技术。
研究艇基低空遥感成像系统的构成与性能,通过分析艇基低空遥感影像数据的特点,对原始影像进行畸变差纠正,去除了镜头光学畸变的影响。在研究目前特征点提取算子和影像匹配算法的基础上,选取稳定性较好的Harris算子,并结合提取精度较高的Forstner算子思想对其改进,对艇基低空遥感影像进行特征点提取,提高了特征点提取精度。采用金字塔影像搜索策略,从粗到精匹配同名像点,最高层采用双向匹配方法,保证初始匹配精度,逐层采用互相关系数匹配法,最底层采用最小二乘精匹配,使匹配同名像点的精度达到子像素级,最后得到同名像点的坐标,可应用于影像自动拼接。
Along with the social informatization, the demands to the real-time and low-cost high-resolution images are increasing rapidly. Unmanned Aerial Vehicle Low Altitude Remote Sensing System (UAVRS) is known as a system with the unmanned airships as its vehicle platforms. UAVRS combines many high technologies such as modern electronics, computer studies, communications, controlling, remote sensing and telemetering with unmanned airship. It possesses the abilities of instant observation to the earth, immediate processing of RS data, as well as acquiring high temporal and spatial resolution images at low cost, which is a very significant and indispensable complementary to traditional aerial and satellite remote sensing. Therefore UAVRS is widely used in Quick Disaster Response, city construction monitoring, precision agriculture, environmental monitoring, investigation of soil utility status and map revision, etc.
With the support of the project“Key Technologies and Application Examples of Geological Survey of Remote Sensing based on the High Resolution Images Obtained by Low Altitude Unmanned Airship”of Ministry of Land and Resources, this paper introduced the preprocessing of low altitude remote sensing images acquired by UAVRS and realized the preprocess of image and a key technology of automatic mosaicking of panoramic images using photogrammetry method, image matching, as a foundation for the further geological analysis.
The preprocessing is an essential phase for the automatic mosaicking of panoramic images. For the reason that the digital cameras in UAVRS are not specially made for Photogrammetry, the elements of interior orientation are unknown, hence the images, which could be affected by the camera distortion and not suitable for complete automatic image mosaicking, need preprocessing. The errors of cameras are mainly caused by the distortion of optical lenses, called optical lens distortion, will lower both the quality of images and accuracy of processing. Therefore, the calibration of optical lens distortion is absolutely necessary in image preprocessing.
The DPMatrix system developed by Wuhan University was used to correct the camera distortion and get the elements of interior orientation as well as distortion factors by using the line mathematical equations. After resampling with bilinear difference method, the image was obtained through distortion correction, thus provides the preparation for the next step.
Image matching, which is based on the image feature points extracting, is a key technology for the automatic mosaicking of panoramic images. At present, the often-used feature extraction operators include Moravec operator, Forstner operator, Harris operator, Trajkovic operator and the SIFT operator. After comparing feature points extraction algorithm’s speed, accuracy and overall flexibility, the Harris operator was found to be of the most stable, robust and the highest accuracy. In this study, the high precision Forstner operator and faster feature extraction Harris operator were combined, and then the Harris improved algorithm was gotten. The Low Altitude Images has high spatial resolution, and in order to improve the running speed and make a Uniform distribution of feature points, not only the information-rich regions were focused on. The original image (2848pixel×4288pixel) was divided into 220 small units (13pixel×19pixel), and for each unit, the feature points were extracted with Harris operator. Eventually, a total of 220 points were Extracted, achieving sub-pixel accuracy.
On the base of extracting image feature points, it’s very important to match corresponding points accurately and fast. Corresponding points were matched in the right image by grayscale matching algorithm under the feature points extracted in the left image. The method of pyramid hierarchical searching was used to enhance operational efficiency and reduce operation time because of the high resolution of the low-altitude unmanned airship’s image. After low-pass filter in the original image, the rough positions of corresponding points were found by Rough correlation, and the results were used as predictive values .Then high-frequency information were added step by step, with the precise correlation in the smaller and smaller areas. For the top layer of the pyramid, a two-way matching was used to ensure the matching accuracy, then the matchings in the middle layers were decided by the correlation coefficient method, at last the matching was refined by the least square in the bottom layer of the pyramid to improve the accuracy of image matching. In the matching process, the features points passed from bottom to top while the matched corresponding points passed in the opposite direction. Eventually, a total of 178 corresponding points with their own coordinates were obtained, providing the solid foundation and Necessary prerequisite for the realization of automatic mosaicking of panoramic images
研究艇基低空遥感成像系统的构成与性能,通过分析艇基低空遥感影像数据的特点,对原始影像进行畸变差纠正,去除了镜头光学畸变的影响。在研究目前特征点提取算子和影像匹配算法的基础上,选取稳定性较好的Harris算子,并结合提取精度较高的Forstner算子思想对其改进,对艇基低空遥感影像进行特征点提取,提高了特征点提取精度。采用金字塔影像搜索策略,从粗到精匹配同名像点,最高层采用双向匹配方法,保证初始匹配精度,逐层采用互相关系数匹配法,最底层采用最小二乘精匹配,使匹配同名像点的精度达到子像素级,最后得到同名像点的坐标,可应用于影像自动拼接。
Along with the social informatization, the demands to the real-time and low-cost high-resolution images are increasing rapidly. Unmanned Aerial Vehicle Low Altitude Remote Sensing System (UAVRS) is known as a system with the unmanned airships as its vehicle platforms. UAVRS combines many high technologies such as modern electronics, computer studies, communications, controlling, remote sensing and telemetering with unmanned airship. It possesses the abilities of instant observation to the earth, immediate processing of RS data, as well as acquiring high temporal and spatial resolution images at low cost, which is a very significant and indispensable complementary to traditional aerial and satellite remote sensing. Therefore UAVRS is widely used in Quick Disaster Response, city construction monitoring, precision agriculture, environmental monitoring, investigation of soil utility status and map revision, etc.
With the support of the project“Key Technologies and Application Examples of Geological Survey of Remote Sensing based on the High Resolution Images Obtained by Low Altitude Unmanned Airship”of Ministry of Land and Resources, this paper introduced the preprocessing of low altitude remote sensing images acquired by UAVRS and realized the preprocess of image and a key technology of automatic mosaicking of panoramic images using photogrammetry method, image matching, as a foundation for the further geological analysis.
The preprocessing is an essential phase for the automatic mosaicking of panoramic images. For the reason that the digital cameras in UAVRS are not specially made for Photogrammetry, the elements of interior orientation are unknown, hence the images, which could be affected by the camera distortion and not suitable for complete automatic image mosaicking, need preprocessing. The errors of cameras are mainly caused by the distortion of optical lenses, called optical lens distortion, will lower both the quality of images and accuracy of processing. Therefore, the calibration of optical lens distortion is absolutely necessary in image preprocessing.
The DPMatrix system developed by Wuhan University was used to correct the camera distortion and get the elements of interior orientation as well as distortion factors by using the line mathematical equations. After resampling with bilinear difference method, the image was obtained through distortion correction, thus provides the preparation for the next step.
Image matching, which is based on the image feature points extracting, is a key technology for the automatic mosaicking of panoramic images. At present, the often-used feature extraction operators include Moravec operator, Forstner operator, Harris operator, Trajkovic operator and the SIFT operator. After comparing feature points extraction algorithm’s speed, accuracy and overall flexibility, the Harris operator was found to be of the most stable, robust and the highest accuracy. In this study, the high precision Forstner operator and faster feature extraction Harris operator were combined, and then the Harris improved algorithm was gotten. The Low Altitude Images has high spatial resolution, and in order to improve the running speed and make a Uniform distribution of feature points, not only the information-rich regions were focused on. The original image (2848pixel×4288pixel) was divided into 220 small units (13pixel×19pixel), and for each unit, the feature points were extracted with Harris operator. Eventually, a total of 220 points were Extracted, achieving sub-pixel accuracy.
On the base of extracting image feature points, it’s very important to match corresponding points accurately and fast. Corresponding points were matched in the right image by grayscale matching algorithm under the feature points extracted in the left image. The method of pyramid hierarchical searching was used to enhance operational efficiency and reduce operation time because of the high resolution of the low-altitude unmanned airship’s image. After low-pass filter in the original image, the rough positions of corresponding points were found by Rough correlation, and the results were used as predictive values .Then high-frequency information were added step by step, with the precise correlation in the smaller and smaller areas. For the top layer of the pyramid, a two-way matching was used to ensure the matching accuracy, then the matchings in the middle layers were decided by the correlation coefficient method, at last the matching was refined by the least square in the bottom layer of the pyramid to improve the accuracy of image matching. In the matching process, the features points passed from bottom to top while the matched corresponding points passed in the opposite direction. Eventually, a total of 178 corresponding points with their own coordinates were obtained, providing the solid foundation and Necessary prerequisite for the realization of automatic mosaicking of panoramic images
引文
[1]马瑞升.微型无人机航空遥感系统及其影像几何纠正研究[D].南京:南京农业大学, 2004.
[2]王聪华.无人飞行器低空遥感影像数据处理方法[D].山东威海:山东科技大学, 2002.
[3]巩丹超.高分辨率卫星遥感立体影像处理模型与算法[D].北京:解放军信息工程大学, 2003.
[4]彭晓东,林宗坚.无人飞艇低空航测系统[J].测绘科学, 2009, 34(4): 11~20.
[5]刘奇志.低空摄影测量技术的发展与应用[J].山东科技大学学报自然科学版, 2007, 26: 150～152.
[6]Josue Jr. G. Ramos, Silvio M. Maeta, Luiz G. B Mirisola, et al. A Software Environment for an Autonomous Unmanned Airship[C]. International Conference on Advanced Intelligent Mechatrinics, 1999.
[7]Josue Jr. G. Ramos, Ely Carneiro de Paiva, Jose Raul Azinheira, et al. Autonomous Flight Experiment with A Robotic Unmanned Airship[C]. International Conference on Robotics & Automation, 2001.
[8]Josue Jr. G. Ramos, Silvio M Maeta, Marcel Bergerman, et al. Development of a VRML/Java Unmanned Airship Simulating Enviroment[C]. International Conference on Intelligent Robots and Systems, 1999.
[9]Sang-Jong Lee, Seong-pil Kim, Tae-Sik Kim, et al. Development of Autonomous Flight Control System for 50m Unmanned Airship[J]. ISSNIP ,2004: 457~462.
[10]Filoktimon Repoulias, Evangelos Papadopoulos. Dynamically Feasible Trajectory and Open-Loop Control Design for Unmanned Airships[C]. 2007 Mediterranean Conference on Control and Automation, 2007.
[11] Jose Raul Azinheira, Ely Carneiro de Paiva, Josue Jr. G. Ramos, et al. Mission Path Following for an Autonomous Unmanned Airship[C]. International Conference on Robotics & Automation, 2000.
[12]D. Gray, M. Nagatsuka, R.Miura. Autopsy of Unmanned Airship Television Broadcast[J]. Yokosuka Radio Communications Research Centre, 2006.
[13]Herwitz R.S. , Lee F.J. , Arvesen J.C. Precision Agriculture as a Commercial Application for Solar-Powered Unmanned Aerial Vehicles[C]. 1st American Institue of Aeronautics and Astronautics UAV Conference, 2002.
[14]Jinjun Rao, Zhenbang Gong, Jun Luo, et al. Unmanned Airships for Emergency Management[C]. International Workshop on Safety, Security and Rescue Robotics, 2005.
[15]Dugan J. and Piotrowski C. Develpomental system for maritime rapid environment assessment using UAVs[C]. 5st ONERA_DLR aerospace symposium, 2003
[16]Robert M.N. Unmanned air vehicles-coming of age at last[J]. Air & space Europe, 2000, 2(1): 23~25.
[17]Toth C. K , G-Brzezinska D. , Merry C. Supporting traffic flow management with high-definition imagery[C]. ISPRS workshop on high resolution mapping from space 2003. Hannover, 2003.
[18]李兵,邱京宪,李和军.无人机摄影测量技术的探索与应用研究[J].北京测绘, 2008, (1): 1~3.
[19]戴斌.航空摄影测量在土地利用现状更新调查中的应用[J].铁道勘察, 2005, (6): 24~26.
[20]申海建,郭荣中,黄小波等.微型无人机(MUAV)航空摄影测量技术在土地整理项目规划设计中的应用[C]. 2007年中国土地学会年会, 2007.
[21]崔洪禹,孙微,牛彦彬等.南极采用飞艇进行航空摄影测量的可行性[J].测绘与空间地理信息, 2005, 28(2): 6~9.
[22]段福州.近地轻型数码航空摄影测量系统研究[D].北京:首都师范大学, 2007.
[23]杜小宇.数字近景摄影测量系统精度分析和控制[D].南京:南京航空航天大学, 2008.
[24]于岱峰,李良良,周广勇.摄影测量特征点提取与匹配技术研究[J].人工智能及识别技术, 2008, 12(20): 503~506.
[25]官云兰,张红军,刘向美.点特征提取算法探讨[J].华东理工学院学报, 2007, 30(1): 42~46.
[26]刘士宽,郭增长,李海启,王志龙,赵盼盼.基于Matlab的点特征提取方法[J].黑龙江科技信息, 2009, (03): 38.
[27]Moravec,H. P. Towards Automatic Visual Obstacle Avoidance[C]. Proceeding of the 5th International Joint Conference on Artificial Intelligence, 1997.
[28]Forstner. W. , E. Gulch. A fast operator for detection and precise location of distinct points,corners and centres of circular features[C]. Interlaken: Switzerland Proceeding of Intercommission Workshop on Fast Processing of Photogrammetric Data, 1987.
[29]Harris, C. , Stephens, M., A combined corner and edge detector[C]. Proceeding of the 4th Alvey Vision Conference, 1988.
[30]张莉,汪大明. Forstner算子及其改进[J].北京工业职业技术学院学报, 2007, 6(3): 17~18.
[31]Trajkovic, M. , Hedley, M. , Fast corner detector[J]. Image and Vision Computing, 1998, (16): 75~87.
[32]唐敏.基于边缘与角点检测的特征提取方法与应用研究[D].北京:国防科学技术大学研究生院, 2006.
[33]Lowe D G. Distinctive Image Feature from Scale-invariant Keypoints[J] International Journal of Computer Vision, 2004, 60 (2): 91-110.
[34]张少辉,沈晓蓉,范耀祖.一种基于图像特征点提取及匹配的方法[J].北京航空航天大学学报, 2008, 34(5): 516~519.
[35]张春美,龚志辉,孙雷.改进SIFT特征在图像匹配中的应用[J].计算机工程与应用, 2008, 44(2): 95~97.
[36]吴铭,林锦国,梅雪.用于图像拼接的特征提取算法研究[J].计算机工程与设计, 2009, 30(2): 440~442.
[37]陈慧.无人飞艇飞行控制系统软件设计[D].北京:中国科学院光电研究院, 2008.
[38]徐柳华.无人飞行器影像处理研究及其实现[D].湖南长沙:中南大学, 2009.
[39]彭永超.低空摄影测量的数据处理与误差分析[J].物探装备, 2008, 18( 2): 78～81.
[40]冯文灏,商浩亮,侯文广.影像的数字畸变模型[J].武汉大学学报(信息科学版), 2006, 31(2): 99~103.
[41]闾海庆.基于非量测数码相机的近景摄影测量数据处理方法研究[D].湖南长沙:中南大学, 2006.
[42]王成亮.基于普通数码影像的近景摄影技术研究与应用[D].湖南长沙:中南大学, 2006.
[43]谢东海,詹总谦,江万寿.改进Harris算子用于点特征的精确定位[J].测绘信息与工程, 2003, 28(2): 22~23.
[44]张登荣,俞乐,蔡志刚.点特征和小波金字塔技术的遥感图像快速匹配技术[J].浙江大学学报(理学版), 2007, 34(4): 465~468.
[45]黄玉琪,牟晓辉,钱曾波.航空摄影立体相对的全自动影像匹配[J].测绘学院学报, 2000,17(4): 277~283.
[46]张剑清,潘励,王树根等.摄影测量学[M].武汉:武汉大学出版社, 2003.
[47]陶本藻.自由网平差[J].工程勘察, 1999, (3): 42~45.
[2]王聪华.无人飞行器低空遥感影像数据处理方法[D].山东威海:山东科技大学, 2002.
[3]巩丹超.高分辨率卫星遥感立体影像处理模型与算法[D].北京:解放军信息工程大学, 2003.
[4]彭晓东,林宗坚.无人飞艇低空航测系统[J].测绘科学, 2009, 34(4): 11~20.
[5]刘奇志.低空摄影测量技术的发展与应用[J].山东科技大学学报自然科学版, 2007, 26: 150～152.
[6]Josue Jr. G. Ramos, Silvio M. Maeta, Luiz G. B Mirisola, et al. A Software Environment for an Autonomous Unmanned Airship[C]. International Conference on Advanced Intelligent Mechatrinics, 1999.
[7]Josue Jr. G. Ramos, Ely Carneiro de Paiva, Jose Raul Azinheira, et al. Autonomous Flight Experiment with A Robotic Unmanned Airship[C]. International Conference on Robotics & Automation, 2001.
[8]Josue Jr. G. Ramos, Silvio M Maeta, Marcel Bergerman, et al. Development of a VRML/Java Unmanned Airship Simulating Enviroment[C]. International Conference on Intelligent Robots and Systems, 1999.
[9]Sang-Jong Lee, Seong-pil Kim, Tae-Sik Kim, et al. Development of Autonomous Flight Control System for 50m Unmanned Airship[J]. ISSNIP ,2004: 457~462.
[10]Filoktimon Repoulias, Evangelos Papadopoulos. Dynamically Feasible Trajectory and Open-Loop Control Design for Unmanned Airships[C]. 2007 Mediterranean Conference on Control and Automation, 2007.
[11] Jose Raul Azinheira, Ely Carneiro de Paiva, Josue Jr. G. Ramos, et al. Mission Path Following for an Autonomous Unmanned Airship[C]. International Conference on Robotics & Automation, 2000.
[12]D. Gray, M. Nagatsuka, R.Miura. Autopsy of Unmanned Airship Television Broadcast[J]. Yokosuka Radio Communications Research Centre, 2006.
[13]Herwitz R.S. , Lee F.J. , Arvesen J.C. Precision Agriculture as a Commercial Application for Solar-Powered Unmanned Aerial Vehicles[C]. 1st American Institue of Aeronautics and Astronautics UAV Conference, 2002.
[14]Jinjun Rao, Zhenbang Gong, Jun Luo, et al. Unmanned Airships for Emergency Management[C]. International Workshop on Safety, Security and Rescue Robotics, 2005.
[15]Dugan J. and Piotrowski C. Develpomental system for maritime rapid environment assessment using UAVs[C]. 5st ONERA_DLR aerospace symposium, 2003
[16]Robert M.N. Unmanned air vehicles-coming of age at last[J]. Air & space Europe, 2000, 2(1): 23~25.
[17]Toth C. K , G-Brzezinska D. , Merry C. Supporting traffic flow management with high-definition imagery[C]. ISPRS workshop on high resolution mapping from space 2003. Hannover, 2003.
[18]李兵,邱京宪,李和军.无人机摄影测量技术的探索与应用研究[J].北京测绘, 2008, (1): 1~3.
[19]戴斌.航空摄影测量在土地利用现状更新调查中的应用[J].铁道勘察, 2005, (6): 24~26.
[20]申海建,郭荣中,黄小波等.微型无人机(MUAV)航空摄影测量技术在土地整理项目规划设计中的应用[C]. 2007年中国土地学会年会, 2007.
[21]崔洪禹,孙微,牛彦彬等.南极采用飞艇进行航空摄影测量的可行性[J].测绘与空间地理信息, 2005, 28(2): 6~9.
[22]段福州.近地轻型数码航空摄影测量系统研究[D].北京:首都师范大学, 2007.
[23]杜小宇.数字近景摄影测量系统精度分析和控制[D].南京:南京航空航天大学, 2008.
[24]于岱峰,李良良,周广勇.摄影测量特征点提取与匹配技术研究[J].人工智能及识别技术, 2008, 12(20): 503~506.
[25]官云兰,张红军,刘向美.点特征提取算法探讨[J].华东理工学院学报, 2007, 30(1): 42~46.
[26]刘士宽,郭增长,李海启,王志龙,赵盼盼.基于Matlab的点特征提取方法[J].黑龙江科技信息, 2009, (03): 38.
[27]Moravec,H. P. Towards Automatic Visual Obstacle Avoidance[C]. Proceeding of the 5th International Joint Conference on Artificial Intelligence, 1997.
[28]Forstner. W. , E. Gulch. A fast operator for detection and precise location of distinct points,corners and centres of circular features[C]. Interlaken: Switzerland Proceeding of Intercommission Workshop on Fast Processing of Photogrammetric Data, 1987.
[29]Harris, C. , Stephens, M., A combined corner and edge detector[C]. Proceeding of the 4th Alvey Vision Conference, 1988.
[30]张莉,汪大明. Forstner算子及其改进[J].北京工业职业技术学院学报, 2007, 6(3): 17~18.
[31]Trajkovic, M. , Hedley, M. , Fast corner detector[J]. Image and Vision Computing, 1998, (16): 75~87.
[32]唐敏.基于边缘与角点检测的特征提取方法与应用研究[D].北京:国防科学技术大学研究生院, 2006.
[33]Lowe D G. Distinctive Image Feature from Scale-invariant Keypoints[J] International Journal of Computer Vision, 2004, 60 (2): 91-110.
[34]张少辉,沈晓蓉,范耀祖.一种基于图像特征点提取及匹配的方法[J].北京航空航天大学学报, 2008, 34(5): 516~519.
[35]张春美,龚志辉,孙雷.改进SIFT特征在图像匹配中的应用[J].计算机工程与应用, 2008, 44(2): 95~97.
[36]吴铭,林锦国,梅雪.用于图像拼接的特征提取算法研究[J].计算机工程与设计, 2009, 30(2): 440~442.
[37]陈慧.无人飞艇飞行控制系统软件设计[D].北京:中国科学院光电研究院, 2008.
[38]徐柳华.无人飞行器影像处理研究及其实现[D].湖南长沙:中南大学, 2009.
[39]彭永超.低空摄影测量的数据处理与误差分析[J].物探装备, 2008, 18( 2): 78～81.
[40]冯文灏,商浩亮,侯文广.影像的数字畸变模型[J].武汉大学学报(信息科学版), 2006, 31(2): 99~103.
[41]闾海庆.基于非量测数码相机的近景摄影测量数据处理方法研究[D].湖南长沙:中南大学, 2006.
[42]王成亮.基于普通数码影像的近景摄影技术研究与应用[D].湖南长沙:中南大学, 2006.
[43]谢东海,詹总谦,江万寿.改进Harris算子用于点特征的精确定位[J].测绘信息与工程, 2003, 28(2): 22~23.
[44]张登荣,俞乐,蔡志刚.点特征和小波金字塔技术的遥感图像快速匹配技术[J].浙江大学学报(理学版), 2007, 34(4): 465~468.
[45]黄玉琪,牟晓辉,钱曾波.航空摄影立体相对的全自动影像匹配[J].测绘学院学报, 2000,17(4): 277~283.
[46]张剑清,潘励,王树根等.摄影测量学[M].武汉:武汉大学出版社, 2003.
[47]陶本藻.自由网平差[J].工程勘察, 1999, (3): 42~45.