基于改进KAZE的无人机航拍图像拼接算法
|
摘要
为了更好地解决航拍图像易受光照、旋转变化、尺度变化等影响,KAZE算法实时性较差以及基于K近邻的特征匹配算法耗时较长等问题,该文提出了一种基于改进KAZE的无人机航拍图像拼接算法.该方法首先利用加速的KAZE算法提取图像的特征点,采用二进制特征描述子FREAK (Fast rctina kcypoint)进行特征点描述,然后使用Crid-KNN算法进行特征点粗匹配,利用随机一致性算法对匹配的特征点进一步提纯并计算几何变换模型,最后采用加权平均算法对图像进行融合.实验结果表明,该文所提算法使图像在光照变化、旋转变化及尺度变化下具有较好的性能,且处理速度较KAZE算法与K近邻特征匹配算法有较大提升,是一种稳定、精确度高、拼接效果良好的无人机航拍图像拼接方法.
The aerial image is subject to many effects including light, rotation changes, changes in dimensions and so on.The real-time performance of the KAZE algorithm is not desirable and the K-nearest neighbor(KNN) match algorithm takes a long time. Therefore, we propose a mosaic algorithm for UAV aerial image based on the improved KAZE. Firstly,we use an accelerated KAZE algorithm to extract feature points of the image, and use the binary feature descriptor fast retina keypoint(FREAK) to describe the feature points. Then, we adopt the Grid-KNN algorithm for rough match of these points, and use the random sample consensus algorithm for exact match and calculating the geometric transform model. Finally, we use the weighted average algorithm for image fusion. Experimental results show that compared with the KAZE algorithm and the KNN algorithm, the proposed algorithm has better performance on changes of illumination,rotation and scale, as well as processing speed. It is a stable, accurate and stitching algorithm.
The aerial image is subject to many effects including light, rotation changes, changes in dimensions and so on.The real-time performance of the KAZE algorithm is not desirable and the K-nearest neighbor(KNN) match algorithm takes a long time. Therefore, we propose a mosaic algorithm for UAV aerial image based on the improved KAZE. Firstly,we use an accelerated KAZE algorithm to extract feature points of the image, and use the binary feature descriptor fast retina keypoint(FREAK) to describe the feature points. Then, we adopt the Grid-KNN algorithm for rough match of these points, and use the random sample consensus algorithm for exact match and calculating the geometric transform model. Finally, we use the weighted average algorithm for image fusion. Experimental results show that compared with the KAZE algorithm and the KNN algorithm, the proposed algorithm has better performance on changes of illumination,rotation and scale, as well as processing speed. It is a stable, accurate and stitching algorithm.
引文
1 Li Z Q, Isler V. Large scale image mosaic construction for agricultural applications. IEEE Robotics and Automation Letters, 2016, 1(1):295-302
2 Li Yan-Shan, Pei Ji-Hong, Xie Wei-Xin, Li Liang-Qun. A new fast automatic mosaic method on unmanned aerial vehicle images, Acta Electronica Sinica, 2012, 40(5):935-940(李岩山,裴继红,谢维信,李良群.一种新的无人机航拍序列图像快速拼接方法.电子学报,2012, 40(5):935-940)
3 Tsai C H, Lin Y C. An accelerated image matching technique for UAV orthoimage registration. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 128:130-145
4 Ghosh D, Kaabouch N. A survey on image mosaicing techniques. Journal of Visual Communication and Image Representation, 2016, 34:1-11
5 Wang Z C, Chen Y F, Zhu Z W, Zhao W D. An automatic panoramic image mosaic method based on graph model. Multimedia Tools and Applications, 2016, 75(5):2725-2740
6 Zhang Gui-Mei, Sun Xiao-Xu, Liu Jian-Xin, Chu Jun. Research on TV-L~1 optical flow model for image registration based on fractional-order differentiation. Acta Automatica Sinica, 2017, 43(12):2213-2224(张桂梅,孙晓旭,刘建新,储珺.基于分数阶微分的TV-L~1光流模型的图像配准方法研究.自动化学报,2017, 43(12):2213-2224)
7 Cai Guo-Rong, Li Shao-Zi, Wu Yun-Dong, Su Song-Zhi,Chen Shui-Li. A perspective invariant image matching algorithm. Acta Automatica Sinica, 2013, 39(7):1053-1061(蔡国榕,李绍滋,吴云东,苏松志,陈水利.一种透视不变的图像匹配笔法.自动化学报,2013, 39(7):1053-1061)
8 Harris C, Stephens M. A combined corner and edge detector. In:Proceedings of the 4th Alvey Vision Conference.Manchester, UK:Alvety Vision Club, 1988. 4-151
9 Smith S M, Brady J M. SUSAN A new approach to low level image processing. International Journal of Computer Vision, 1997, 23(1):45-78
10 Lowe D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004,60(2):91-110
11 Yan Xue-Jun, Zhao Chun-Xia, Yuan Xia. 2DPCA-SIFT:an efficient local feature descriptor. Acta Automatica Sinica,2014, 40(4):675-682(颜雪军,赵春霞,袁夏.2DPCA-SIFT:一种有效的局部特征描述方法.自动化学报,2014, 40(4):675-682)
12 Bay H, Ess A, Tuytelaars T, Van Gool L. Speeded-up robust features(SURF). Computer Vision and Image Understanding, 2008, 110(3):346-359
13 Yan Zi-Geng, Jiang Jian-Guo, Guo Dan. Image matching based on SURF feature and Delaunay triangular meshes.Acta Automatica Sinica, 2014, 40(6):1216-1222(闫自庚,蒋建国,郭丹.基于SURF特征和Delaunay三角网格的图像匹配.自动化学报,2014, 40(6):1216-1222)
14 Rublee E, Rabaud V, Konolige K, Bradski G. ORB:an efficient, alternative to SIFT or SURF. In:Proceedings of the2011 IEEE International Conference on Computer Vision.Barcelona, Spain:IEEE, 2011. 2564-2571
15 Du Cheng-Yao, Yuan Jing-Ling, Chen Min-Cheng, Li Tao.Real-time panoramic video stitching based on GPU acceleration using Local ORB feature extraction. Journal of Computer Research and Development, 2017, 54(6):1316-1325(杜承垚,袁景凌,陈旻骋,李涛.GPU加速与L-ORB特征提取的全景视频实时拼接.计算机研究与发展,2017, 54(6):1316-1325)
16 Leutenegger S, Chli M, Siegwart R Y. BRISK:binary robust invariant scalable keypoints. In:Proceedings of 2011 IEEE International Conference on Computer Vision. Barcelona,Spain:IEEE, 2011. 2548-2555
17 Dong Qiang, Liu Jing-Hong, Wang Chao, Zhou Qian-Fei.Image mosaic algorithm based on improved BRISK. Journal of Electronics&Information Technology, 2017, 39(2):444-450(董强,刘晶红,王超,周前飞.基于改进BRISK的图像拼接算法.电子与信息学报,2017, 39(2):444-450)
18 Alcantarilla P F, Bartoli A, Davison A J. KAZE features. In:Proceedings of the 12th European Conference on Computer Vision. Florence, Italy:Springer, 2012. 214-227
19 Mukherjee P, Lall B. Saliency and KAZE features assisted object segmentation. Image and Vision Computing, 2017,61:82-97
20 Alcantarilla P, Nuevo J, Bartoli A. Fast explicit diffusion for accelerated features in nonlinear scale spaces. In:Proceedings of British Machine Vision Conference. Bristol, England,UK:BMVC, 2013. 13.1-13.11
21 Alahi A, Ortiz R, Vandergheynst P. Freak:fast retina keypoint. In:Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI,USA:IEEE, 2012. 510-517
22 Ma X M, Liu D, Zhang J, Xin J. A fast affine-invariant features for image stitching under large viewpoint changes.Neurocomputing, 2015, 151:1430-1438
23 Tuytelaars T, Mikolajczyk K. Local invariant feature detectors:a survey. Foundations and Trends in Computer Graphics and Vision, 2008, 3(3):177-280
24 Bian J W, Lin W Y, Matsushita Y, Yeung S T, Nguyen T D, Cheng M M. GMS:grid-based motion statistics for fast, ultra-robust feature correspondence. In:Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA:IEEE, 2017. 2828-2837
2 Li Yan-Shan, Pei Ji-Hong, Xie Wei-Xin, Li Liang-Qun. A new fast automatic mosaic method on unmanned aerial vehicle images, Acta Electronica Sinica, 2012, 40(5):935-940(李岩山,裴继红,谢维信,李良群.一种新的无人机航拍序列图像快速拼接方法.电子学报,2012, 40(5):935-940)
3 Tsai C H, Lin Y C. An accelerated image matching technique for UAV orthoimage registration. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 128:130-145
4 Ghosh D, Kaabouch N. A survey on image mosaicing techniques. Journal of Visual Communication and Image Representation, 2016, 34:1-11
5 Wang Z C, Chen Y F, Zhu Z W, Zhao W D. An automatic panoramic image mosaic method based on graph model. Multimedia Tools and Applications, 2016, 75(5):2725-2740
6 Zhang Gui-Mei, Sun Xiao-Xu, Liu Jian-Xin, Chu Jun. Research on TV-L~1 optical flow model for image registration based on fractional-order differentiation. Acta Automatica Sinica, 2017, 43(12):2213-2224(张桂梅,孙晓旭,刘建新,储珺.基于分数阶微分的TV-L~1光流模型的图像配准方法研究.自动化学报,2017, 43(12):2213-2224)
7 Cai Guo-Rong, Li Shao-Zi, Wu Yun-Dong, Su Song-Zhi,Chen Shui-Li. A perspective invariant image matching algorithm. Acta Automatica Sinica, 2013, 39(7):1053-1061(蔡国榕,李绍滋,吴云东,苏松志,陈水利.一种透视不变的图像匹配笔法.自动化学报,2013, 39(7):1053-1061)
8 Harris C, Stephens M. A combined corner and edge detector. In:Proceedings of the 4th Alvey Vision Conference.Manchester, UK:Alvety Vision Club, 1988. 4-151
9 Smith S M, Brady J M. SUSAN A new approach to low level image processing. International Journal of Computer Vision, 1997, 23(1):45-78
10 Lowe D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004,60(2):91-110
11 Yan Xue-Jun, Zhao Chun-Xia, Yuan Xia. 2DPCA-SIFT:an efficient local feature descriptor. Acta Automatica Sinica,2014, 40(4):675-682(颜雪军,赵春霞,袁夏.2DPCA-SIFT:一种有效的局部特征描述方法.自动化学报,2014, 40(4):675-682)
12 Bay H, Ess A, Tuytelaars T, Van Gool L. Speeded-up robust features(SURF). Computer Vision and Image Understanding, 2008, 110(3):346-359
13 Yan Zi-Geng, Jiang Jian-Guo, Guo Dan. Image matching based on SURF feature and Delaunay triangular meshes.Acta Automatica Sinica, 2014, 40(6):1216-1222(闫自庚,蒋建国,郭丹.基于SURF特征和Delaunay三角网格的图像匹配.自动化学报,2014, 40(6):1216-1222)
14 Rublee E, Rabaud V, Konolige K, Bradski G. ORB:an efficient, alternative to SIFT or SURF. In:Proceedings of the2011 IEEE International Conference on Computer Vision.Barcelona, Spain:IEEE, 2011. 2564-2571
15 Du Cheng-Yao, Yuan Jing-Ling, Chen Min-Cheng, Li Tao.Real-time panoramic video stitching based on GPU acceleration using Local ORB feature extraction. Journal of Computer Research and Development, 2017, 54(6):1316-1325(杜承垚,袁景凌,陈旻骋,李涛.GPU加速与L-ORB特征提取的全景视频实时拼接.计算机研究与发展,2017, 54(6):1316-1325)
16 Leutenegger S, Chli M, Siegwart R Y. BRISK:binary robust invariant scalable keypoints. In:Proceedings of 2011 IEEE International Conference on Computer Vision. Barcelona,Spain:IEEE, 2011. 2548-2555
17 Dong Qiang, Liu Jing-Hong, Wang Chao, Zhou Qian-Fei.Image mosaic algorithm based on improved BRISK. Journal of Electronics&Information Technology, 2017, 39(2):444-450(董强,刘晶红,王超,周前飞.基于改进BRISK的图像拼接算法.电子与信息学报,2017, 39(2):444-450)
18 Alcantarilla P F, Bartoli A, Davison A J. KAZE features. In:Proceedings of the 12th European Conference on Computer Vision. Florence, Italy:Springer, 2012. 214-227
19 Mukherjee P, Lall B. Saliency and KAZE features assisted object segmentation. Image and Vision Computing, 2017,61:82-97
20 Alcantarilla P, Nuevo J, Bartoli A. Fast explicit diffusion for accelerated features in nonlinear scale spaces. In:Proceedings of British Machine Vision Conference. Bristol, England,UK:BMVC, 2013. 13.1-13.11
21 Alahi A, Ortiz R, Vandergheynst P. Freak:fast retina keypoint. In:Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI,USA:IEEE, 2012. 510-517
22 Ma X M, Liu D, Zhang J, Xin J. A fast affine-invariant features for image stitching under large viewpoint changes.Neurocomputing, 2015, 151:1430-1438
23 Tuytelaars T, Mikolajczyk K. Local invariant feature detectors:a survey. Foundations and Trends in Computer Graphics and Vision, 2008, 3(3):177-280
24 Bian J W, Lin W Y, Matsushita Y, Yeung S T, Nguyen T D, Cheng M M. GMS:grid-based motion statistics for fast, ultra-robust feature correspondence. In:Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA:IEEE, 2017. 2828-2837