基于深度和彩色双信息特征源的Kinect植物图像拼接
详细信息    查看全文 | 推荐本文 |
  • 英文篇名:Plant image mosaic based on depth and color dual information feature source from Kinect
  • 作者:沈跃 ; 朱嘉慧 ; 刘慧 ; 孙力
  • 英文作者:Shen Yue;Zhu Jiahui;Liu Hui;Sun Li;School of Electrical and Information Engineering, Jiangsu University;School of Food and Biological Engineering, Jiangsu University;
  • 关键词:图像处理 ; 特征提取 ; 图像融合 ; 深度数据 ; 双信息特征源 ; Kinect
  • 英文关键词:image processing;;feature extraction;;image fusion;;depth data;;dual information feature source;;Kinect
  • 中文刊名:NYGU
  • 英文刊名:Transactions of the Chinese Society of Agricultural Engineering
  • 机构:江苏大学电气信息工程学院;江苏大学食品与生物工程学院;
  • 出版日期:2018-03-08
  • 出版单位:农业工程学报
  • 年:2018
  • 期:v.34;No.332
  • 基金:国家自然科学基金项目(51505195);; 江苏省国际科技合作项目(BZ2017067);; 江苏高校优势学科(PAPD)
  • 语种:中文;
  • 页:NYGU201805023
  • 页数:7
  • CN:05
  • ISSN:11-2047/S
  • 分类号:184-190
摘要
图像拼接对制作全景图具有重要作用,传统SIFT(scale-invariant feature transform)图像拼接方法受光照不均匀或风吹影响,存在错位和缺失等情况。针对这一问题,该文提出一种基于Kinect彩色和深度双信息特征源的图像拼接方法。首先对获取的植株彩色图像采用SIFT算法进行特征点提取和特征点匹配,利用Kinect收集到的植株深度数据去除误匹配,采用RANSAC算法寻找投影变换矩阵,最后通过最佳缝合线算法进行图像融合。室内和室外试验结果表明,该文基于Kinect彩色和深度双信息特征源的图像拼接方法能够有效克服光照、风吹等环境因素的影响,避免了图像缺失、亮暗差异、重影等拼接错误,该文方法图像拼接时间较短,平均匹配准确率达96.0%,较传统SIFT图像拼接方法平均准确率提高了5.9%。
        In agricultural production, image mosaic plays an important role for a panorama view when images are collected by on-board real-time sensor. Most of the agricultural operation is outside, and traditional method of image mosaic based on SIFT(scale-invariant feature transform) algorithm is affected by uneven illumination or wind and some other factors, so image after mosaic presents dislocation or absence of information and other mistakes. It is difficult to meet the requirements of the reliability of agricultural vehicle applications by using image feature element method for image mosaic. Aiming at this problem, the paper proposes a new method of image mosaic based on the color and depth dual information feature source from Kinect. Firstly, depth information and color images are collected by Kinect sensor from 3 different angels. Kinect sensor is fixed on the slider, and when the motion slider moves at a constant speed of 0.5 m/s, the Kinect moves at the same speed. The Kinect is invoked to obtain color image and depth information and save them every 0.6 s through MATLAB. Three consecutive time points are selected in order, and the color images of 3 angles are taken as the mosaic images. Secondly, SIFT algorithm is used to get feature points from color images. SIFT algorithm can extract the feature points which have invariance for illumination, affine and projection transformation. It is helpful to reduce the number of feature points matching and improve the speed and accuracy of feature point matching. Thirdly, feature points matches are gotten by similarity measure, and the Euclidean distance with high efficiency is used as the similarity measure in the 2 pictures in this paper. A key point in an image is gotten, and the closest 2 key points in another image are found. If the ratio of the nearest distance to the next closest distance is less than a certain threshold, then the pair of matching points are received. But some wrong matches exist with this method. Too many mismatches may result in mosaic errors. Therefore, a solution is needed to remove mismatches to improve the accuracy of the matches. From the nature of Kinect, if Kinect moves horizontally, the depth data of a fixed point is the same. Based on this characteristic, some mismatches would be removed. If the depth data of these 2 feature points are the same, the match is retained, and otherwise they would be removed. As a result, the accuracy of matches is improved. Next, by RANSAC(random sample consensus) algorithm, projection transformation matrix can be found. The RANSAC algorithm uses the least possible points to estimate the model and then as far as possible to expand scope of the influence of the model. The projection transformation matrix is more accurate than traditional image mosaic method on account of the removing of mismatches. At last, through the best suture line algorithm, image fusion is relatively smooth. From indoor and outdoor test, the mosaic method based on color and depth dual information feature source has obvious advantages, and it can effectively overcome the influence of light, wind and other environmental factors and avoid mosaic errors such as the loss of image and the difference of brightness. In the indoor test, the mosaic method of this article takes 9.70 s, the accuracy of matches is 92.9%, while traditional method based on SIFT algorithm takes 13.04 s, the accuracy of matches is 88.1%. In the outdoor test, the mosaic method of this article takes 71.15 s, the accuracy of matches is 99.1%, while traditional method based on SIFT algorithm takes 77.67 s, the accuracy of matches is 92.1%. So the mosaic method in this article takes less time than the traditional method based on SIFT algorithm. The data of mosaic accuracy show that the average matching accuracy of the method in this article is 96.0%, and the average accuracy is 5.9% higher than the traditional image mosaic method based on SIFT. So, this method can be further applied in other occasions of image mosaic. It can realize precise spraying of drug fertilizers and the control of pests and diseases based on information collected by Kinect.
引文
[1]Guo S,Sun S,Guo J.The application of image mosaic in information collecting for an amphibious spherical robot system[C].IEEE International Conference on Mechatronics and Automation.IEEE,2016:1547-1552.
    [2]Li Z,Isler V.Large scale image mosaic construction for agricultural applications[J].IEEE Robotics&Automation Letters,2016,1(1):295-302.
    [3]Huo C,Men G,Zhao L H.Research on the auxiliary panoramic parking technology based on fast image mosaic[C].International Conference on Informative and Cybernetics for Computational Social Systems.IEEE,2016:216-219.
    [4]Burdea G,Coiffet P.Virtual Reality Technology[M].北京:电子工业出版社,2003.
    [5]Zhang J,Tao B,Liu H,et al.A mosaic method based on feature matching for side scan sonar images[C].Ocean Acoustics.IEEE,2016:1-6.
    [6]张宝龙,李洪蕊,李丹,等.一种针对车载全景系统的图像拼接算法的仿真[J].电子与信息学报,2015,37(5):1149-1153.Zhang Baolong,Li Hongrui,Li Dan,et al.A simulation of image mosaic algorithm based on vehicle panorama system[J].Journal of Electronics and Information Technology,2015,37(5):1149-1153.(in Chinese with English abstract)
    [7]Guo S,Sun S,Guo J.The application of image mosaic in information collecting for an amphibious spherical robot system[C].IEEE International Conference on Mechatronics and Automation.IEEE,2016:1547-1552.
    [8]Liu S.The research and improvement of image mosaicing algorithm and its application in frescoes[C].International Conference on Intelligent Computation Technology and Automation.IEEE,2015:817-820.
    [9]马晓丹,孟庆宽,张丽娇,等.图像拼接重建苹果树冠层器官三维形态[J].农业工程学报,2014,30(12):154-162.Ma Xiaodan,Meng Qingkuan,Zhang Lijiao,et al.Image mosaics reconstruction of canopy organ morphology of apple trees[J].Transactions of the Chinese Society of Agricultural Engineering(Transactions of the CSAE),2014,30(12):154-162.(in Chinese with English abstract)
    [10]周志艳,闫梦璐,陈盛德,等.Harris角点自适应检测的水稻低空遥感图像配准与拼接算法[J].农业工程学报,2015,31(14):186-193.Zhou Zhiyan,Yan Menglu,Chen Shengde,et al.Image registration and stitching algorithm of rice low-altitude remote sensing based on harris corner self-adaptive detection[J].Transactions of the Chinese Society of Agricultural Engineering(Transactions of the CSAE),2015,31(14):186-193.(in Chinese with English abstract)
    [11]姚立健,周高峰,倪忠进,等.基于尺度不变特征转换算子的水果表面图像拼接方法[J].农业工程学报,2015,31(9):161-166.Yao Lijian,Zhou Gaofeng,Ni Zhongjin,et al.Matching method for fruit surface image based on scale invariant feature transform algorithm[J].Transactions of the Chinese Society of Agricultural Engineering(Transactions of the CSAE),2015,31(9):161-166.(in Chinese with English abstract)
    [12]何东健,邵小宁,王丹,等.Kinect获取植物三维点云数据的去噪方法[J].农业机械学报,2016,47(1):331-336.He Dongjian,Shao Xiaoning,Wang Dan,et al.Denoising method of 3D point cloud data of plants obtained by kinect[J].Transactions of The Chinese Society of Agricultural Machinery,2016,47(1):331-336.(in Chinese with English abstract)
    [13]袁晓菲.一种基于kinect深度传感器的点云拼接算法研究[D].沈阳:辽宁大学,2016.Yuan Xiaofei.An Research on Point Cloud Registration Algorithm Based on Kinect Device[D].Shenyang:Liaoning University,2016.(in Chinese with English abstract)
    [14]Khoshelham K,Elberink S O.Accuracy and resolution of Kinect depth data for indoor mapping application[J].Sensors,2012,12(2):1437-1454.
    [15]Bonny M Z,Uddin M S.Feature-based image stitching algorithms[C].International Workshop on Computational Intelligence.IEEE,2017:198-203.
    [16]Wang T,Lu G,Xia Y.An efficient preprocessing method for feature based image stitching[C].International Conference on Wireless Communications&Signal Processing.IEEE,2016:1-5.
    [17]Zhen Y,Sun Z,Li J,et al.An airborne remote sensing image mosaic algorithm based on feature points[C].Sixth International Conference on Instrumentation&Measurement,Computer,Communication and Control.IEEE,2016:202-205.
    [18]陈月,赵岩,王世刚.图像局部特征自适应的快速SIFT图像拼接方法[J].中国光学,2016,9(4):415-421.Chen Yue,Zhao Yan,Wang Shigang.Fast image stitching method based on SIFT with adaptive local image feature[J].Chinese Optics,2016,9(4):415-422.(in Chinese with English abstract)
    [19]陈彦彤.基于局部不变特征的遥感图像星上目标识别技术研究[D].长春:中国科学院长春光学精密机械与物理研究[D].所,2017.Chen Yantong.Research on Satellite Image On-board Target Recognition Based on Local Invariant Feature[D].Changchun:Changchun Institute of Optics,Fine Mechanics and Physics,Chinese Academy of Sciences,2017.(in Chinese with English abstract)
    [20]Barba Ana Victoria,Rosales Silva A Jorge,Perez Hernandez L Manuel.SIFT-SURF commutation using fuzzy logic to image mosaicking[C].Electrical Engineering,Computing Science and Automatic Control,2017,28(3):1-8.
    [21]赵岩,陈月,王世刚.结合投影误差校正的快速SIFT图像拼接[J].光学精密工程,2017,25(6):1645-1651.Zhao Yan,Chen Yue,Wang Shigang.Corrected fast SIFTimage stitching method by combining projection error[J].Optics and Precision Engineering,2017,25(6):1645-1651.(in Chinese with English abstract)
    [22]文伟东,张明.基于SIFT算法的全景图像拼接技术研究[J].计算机系统应用,2017,26(7):227-231.Wen Weidong,Zhang Ming.Research on panoramic image mosaic technology based on SIFT algorithm[J].Computer Systems&Applications,2017,26(7):227-231.(in Chinese with English abstract)
    [23]Tong G,Liu H.Deep sea image mosaic based on SIFTalgorithm[C].International Conference on Information Science and Control Engineering.2017:254-257.
    [24]杨安园,朱悦云.多尺度SIFT特征匹配的足迹图像拼接方法[J].时代农机,2017(1):52-53.Yang Anyuan,Zhu Yueyun.Method of footprint image stitching based on multiple dimensioned SIFT feature matching[J].Times Agricultural Machinery,2017(1):52-53.(in Chinese with English abstract)
    [25]贾银江,徐哲男,苏中滨,等.基于优化SIFT算法的无人机遥感作物影像拼接[J].农业工程学报,2017,33(10):123-129.Jia Yinjiang,Xu Zhenan,Su Zhongbin,et al.Mosaic of crop remote sensing images from UAV based on improved SIFTalgorithm[J].Transactions of the Chinese Society of Agricultural Engineering(Transactions of the CSAE),2017,33(10):123-129.(in Chinese with English abstract)
    [26]张勇,王志锋,马文.基于改进SIFT特征点匹配的图像拼接算法研究[J].微电子学与计算机,2016,33(3):60-64.Zhang Yong,Wang Zhifeng,Ma Wen.Research image mosaic algorithm based on improved SIFT feature matching[J].Microelectronics&Computer,2016,33(3):60-64.(in Chinese with English abstract)
    [27]马强,项昭保,黄良学,等.基于改进SIFT和RANSAC图像拼接算法研究[J].计算机技术与发展,2016,26(4):61-65.Ma Qiang,Xiang Zhaobao,Huang Liangxue,et al.Research on panorama image mosaic algorithm based on improved SIFTand ransac[J].Computer Technology And Development,2016,26(4):61-65.(in Chinese with English abstract)
    [28]Zhen Y,Sun Z,Li J,et al.An airborne remote sensing image mosaic algorithm based on feature points[C].Sixth International Conference on Instrumentation&Measurement,Computer,Communication and Control.IEEE,2016:202-205.
    [29]朱镕杰,朱颖汇,王玲,等.基于尺度不变特征转换算法的棉花双目视觉定位技术[J].农业工程学报,2016,32(6):182-188.Zhu Rongjie,Zhu Yinghui,Wang Ling,et al.Cotton positioning technique based on binocular vision with implementation of scale-invariant feature transform algorithm[J].Transactions of the Chinese Society of Agricultural Engineering(Transactions of the CSAE),2016,32(6):182-188.(in Chinese with English abstract)
    [30]Hossein-Nejad Z,Nasri M.Image registration based on SIFTfeatures and adaptive ransac transform[C].International Conference on Communication and Signal Processing.IEEE,2016.
    [31]Shuang D,Cai H,Yang Y,et al.Analysis of image stitching error based on scale invariant feature transform and random sample consensus[C].International Conference on Fuzzy Systems and Knowledge Discovery.IEEE,2016:1883-1887.
    [32]Redzuwan R,Radzi N A M,Din N M,et al.Affine versus projective transformation for SIFT and RANSAC image matching methods.[J].Operations Research Letters,2011,39(6):406-410.
    [33]夏克付,李鹏飞,陈小平.基于改进RANSAC算法的单应矩阵鲁棒估计方法[J].计算机工程与应用,2017,53(23):147-152.Xia Kefu,Li Pengfei,Chen Xiaoping.Robust method for homography matrix estimation based on improved ransac algorithm.Computer Engineering and Applications[J].Computer Engineering and Applications,2017,53(23):147-152.(in Chinese with English abstract)
    [34]Li Aiguo,Zhou Shuai,Wang Rui.An improved method for eliminating ghosting in image stitching[C].Intelligent Human-Machine Systems and Cybernetic,2017:415-418
    [35]方贤勇,潘志庚,徐丹.图像拼接的改进算法[J].计算机辅助设计与图形学学报,2003,15(11):1362-1365.Fang Xianyong,Pan Zhigeng,Xu Dan.An improved algorithm for image mosaics[J].Journal of Computer-aided Design&Computer Graphics,2003,15(11):1362-1365.(in Chinese with English abstract)
    [36]谷雨,周阳,任刚,等.结合最佳缝合线和多分辨率融合的图像拼接[J].中国图象图形学报,2017,22(6):842-851.Gu Yu,Zhou Yang,Ren Gang,et al.Image stitching by combining optimal seam and multi-resolution fusion[J].Journal of Image and Graphics,2017,22(6):842-851.(in Chinese with English abstract)

© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号

地址:北京市海淀区学院路29号 邮编:100083

电话:办公室:(+86 10)66554848;文献借阅、咨询服务、科技查新:66554700