基于RGB-D相机的油菜分枝三维重构与角果识别定位
|
摘要
为实现高效低成本的油菜植株三维建模和表型参数在线测量,提出一种基于RGB-D相机的油菜分枝三维重建和角果识别定位方法。使用Kinect传感器拍摄角果期油菜分枝在4个视角下的彩色图像和深度图像,进而获取油菜植株的三维点云并滤波。对配准的点云进行旋转变换,计算点云的曲面法矢量和曲率,并由曲率相近的点构成配对点对,再使用基于KD-tree搜索的最近点迭代(ICP)算法实现点云的初配准。将初配准误差作为参考值,调整ICP算法的对应点距离阈值,使用初配准的操作流程对初配准得到的新点云进行再次配准,完成精配准。结合该三维重建方法和针对性的彩色图像处理方法,得到去除主茎的单分枝油菜角果的完整点云,再进行欧氏聚类实现单个角果的空间定位。实验结果表明,提出的三维重建方法具有较强的实时性和鲁棒性,单个角果的三维形态清晰可见,点云平均距离误差小于0. 48 mm,角果总体识别正确率不小于96. 76%。
In order to achieve high-efficiency and low-cost 3 D modeling of rapeseed plant and online measurement of phenotypic parameters,a method for rape branch 3 D reconstruction and pod identification was proposed by using RGB-D camera. A Kinect sensor was adopted to gather the color and depth images of rape branch from four angles,and then the 3 D point cloud of the branch was gained and filtered. A rotation transformation was performed on the point cloud that needed to be registered,and the surface normal vector and curvature of the point cloud were calculated. Point pairs were constituted the points with similar curvature,which were used for the initial registration of the point cloud by using the nearest point iterative( ICP) algorithm based on KD-tree search. Employing the reference value came from the initial registration error,the corresponding point distance threshold of ICP algorithm was adjusted,and then the new point cloud obtained by the initial registration was precisely registered by using the initial registration operation. Combined with the proposed 3 D reconstruction and the specific color image processing,an integrated point cloud of the rape branch without the main stem was produced,and then the single rape pod was identified with the Euclidean distance clustering algorithm. The experiment results showed that the proposed method had good robust and real-time,the 3 D structure of single pod was clearly visible,the average distance error of the point cloud was less than 0. 48 mm,and the overall recognition rate of the pod was no less than 96. 76%.
In order to achieve high-efficiency and low-cost 3 D modeling of rapeseed plant and online measurement of phenotypic parameters,a method for rape branch 3 D reconstruction and pod identification was proposed by using RGB-D camera. A Kinect sensor was adopted to gather the color and depth images of rape branch from four angles,and then the 3 D point cloud of the branch was gained and filtered. A rotation transformation was performed on the point cloud that needed to be registered,and the surface normal vector and curvature of the point cloud were calculated. Point pairs were constituted the points with similar curvature,which were used for the initial registration of the point cloud by using the nearest point iterative( ICP) algorithm based on KD-tree search. Employing the reference value came from the initial registration error,the corresponding point distance threshold of ICP algorithm was adjusted,and then the new point cloud obtained by the initial registration was precisely registered by using the initial registration operation. Combined with the proposed 3 D reconstruction and the specific color image processing,an integrated point cloud of the rape branch without the main stem was produced,and then the single rape pod was identified with the Euclidean distance clustering algorithm. The experiment results showed that the proposed method had good robust and real-time,the 3 D structure of single pod was clearly visible,the average distance error of the point cloud was less than 0. 48 mm,and the overall recognition rate of the pod was no less than 96. 76%.
引文
[1]李岚涛,任涛,汪善勤,等.基于角果期高光谱的冬油菜产量预测模型研究[J/OL].农业机械学报,2017,48(3):221-229.LI Lantao,REN Tao,WANG Shanqin,et al.Prediction models of winter oilseed rape yield based on hyperspectral data at podfilling stage[J/OL].Transactions of the Chinese Society for Agricultural Machinery,2017,48(3):221-229.http:∥www.jcsam.org/jcsam/ch/reader/view_abstract.aspx?flag=1&file_no=20170328&journal_id=jcsam.DOI:10.6041/j.issn.1000-1298.2017.03.028.(in Chinese)
[2]FAN Zhenqi,SI Jingchun,HAN Xu,et al.Design mode for component-based virtual plant software[J].Journal Agricultural Science&Technology,2012,13(4):901-903.
[3]李诗锐,李琪,李海洋,等.基与Kinect V2的实时精确三维重建系统[J].软件学报,2016,27(10):2519-2529.LI Shirui,LI Qi,LI Haiyang,et al.Real-time accurate 3D reconstruction based on Kinect V2[J].Journal of Software,2016,27(10):2519-2529.(in Chinese)
[4]PAULUS S,SCHUMANN H,KUHLUMANN H,et al.High-precision laser scanning system for capturing 3D plant architecture and analysing growth of cereal plants[J].Biosystems Engineering,2014,121:1-11.
[5]YAO Yidan,LIN Chengda,ZHAI Ruifang,et al.Rapeseed 3D reconstruction and morphological parameter measurement based on laser point cloud[C]∥International Conference on Agro-Geoinformatics,2016.
[6]张伟洁,刘刚,郭彩玲,等.基于三维点云的苹果树叶片三维重建研究[J/OL].农业机械学报,2017,48(增刊):103-109.ZHANG Weijie,LIU Gang,GUO Cailing,et al.Apple tree leaf three-dimensional reconstruction based on point cloud[J/OL].Transactions of the Chinese Society for Agricultural Machinery,2017,48(Supp.):103-109.http:∥www.j-csam.org/jcsam/ch/reader/view_abstract.aspx?flag=1&file_no=2017s017&journal_id=jcsam.DOI:10.6041/j.issn.1000-1298.2017.S0.017.(in Chinese)
[7]XIONG Xiong,YU Lejun,YANG Wanneng,et al.A high-throughput stereo-imaging system for quantifying rape leaf traits during the seedling stage[J].Plant Methods,2017,13(1):7.
[8]王传宇,郭新宇,吴升,等.基于计算机视觉的玉米果穗三维重建方法[J/OL].农业机械学报,2014,45(9):274-279.WANG Chuanyu,GUO Xinyu,WU Sheng,et al.Three dimensional reconstruction of maize ear based on computer vision[J/OL].Transactions of the Chinese Society for Agricultural Machinery,2014,45(9):274-279.http:∥www.j-csam.org/jcsam/ch/reader/view_abstract.aspx?flag=1&file_no=20140944&journal_id=jcsam.DOI:10.6041/j.issn.1000-1298.2014.09.044.(in Chinese)
[9]张艳超,庄载椿,肖宇钊,等.基于运动恢复结构算法的油菜NDVI三维分布[J].农业工程学报,2015,31(17):207-214.ZHANG Yanchao,ZHUANG Zaichun,XIAO Yuzhao,et al.Rape plant NDVI 3D distribution based on structure from motion[J].Transactions of the CSAE,2015,31(17):207-214.(in Chinese)
[10]FANG Yihang,LIN Chengda,ZHAI Ruifang,et al.A new approach for measuring 3D digitalized rape leaf parameters based on images[C]∥International Conference on Agro-Geoinformatics,2014:1-5.
[11]宗泽,郭彩玲,张雪,等.基于深度相机的玉米株型参数提取方法研究[J/OL].农业机械学报,2015,46(增刊):50-56.ZONG Ze,GUO Cailing,ZHANG Xue,et al.Maize plant type parameters extraction based on depth camera[J/OL].Transactions of the Chinese Society for Agricultural Machinery,2015,46(Supp.):50-56.http:∥www.j-csam.org/jcsam/ch/reader/view_abstract.aspx?flag=1&file_no=2015S009&journal_id=jcsam.DOI:10.6041/j.issn.1000-1298.2015.S0.009.(in Chinese)
[12]SALAU J,HAAS J H,JUNGE W,et al.Automated calculation of udder depth and rear leg angle in Holstein-Friesian cows using a multi-Kinect cow scanning system[J].Biosystems Engineering,2017,160:154-169.
[13]麦春艳,郑立华,孙红,等.基于RGB-D相机的果树三维重构与果实识别定位[J/OL].农业机械学报,2015,46(增刊):35-40.MAI Chunyan,ZHENG Lihua,SUN Hong,et al.Research on 3D reconstruction of fruit tree and fruit recognition and location method based on RGB-D camera[J/OL].Transactions of the Chinese Society for Agricultural Machinery,2015,46(Supp.):35-40.http:∥www.j-csam.org/jcsam/ch/reader/view_abstract.aspx?flag=1&file_no=2015S006&journal_id=jcsam.DOI:10.6041/j.issn.1000-1298.2015.S0.006.(in Chinese)
[14]VITZRABIN E,EDAN Y.Adaptive thresholding with fusion using a RGB-D sensor for red sweet-pepper detection[J].Biosystems Engineering,2016,146:45-56.
[15]GUO Kaiwen,XU Feng,YU Tao,et al.Real-time geometry,albedo,and motion reconstruction using a single RGB-D camera[J].ACM Transactions on Graphics,2017,36(3):32.
[16]JUAN R,TERVEN D,MARGARITA C E,et al.A multiple camera calibration and point cloud fusion tool for Kinect V2[J].Science of Computer Programming,2017,143:1-8.
[17]郑立华,麦春艳,廖崴,等.基于Kinect相机的苹果树三维点云配准[J/OL].农业机械学报,2016,47(5):9-14.ZHENG Lihua,MAI Chunyan,LIAO Wei,et al.3D point cloud registration for apple tree based on Kinect camera[J/OL].Transactions of the Chinese Society for Agricultural Machinery,2016,47(5):9-14.http:∥www.j-csam.org/jcsam/ch/reader/view_abstract.aspx?flag=1&file_no=20160502&journal_id=jcsam.DOI:10.6041/j.issn.1000-1298.2016.05.002.(in Chinese)
[18]沈跃,潘成凯,刘慧,等.基于改进SIFT-ICP算法的Kinect植株点云配准方法[J/OL].农业机械学报,2017,48(12):183-189.SHEN Yue,PAN Chengkai,LIU Hui,et al.Method of plant point cloud registration based on Kinect of improved SIFT-ICP[J/OL].Transactions of the Chinese Society for Agricultural Machinery,2017,48(12):183-189.http:∥www.j-csam.org/jcsam/ch/reader/view_abstract.aspx?flag=1&file_no=20171221&journal_id=jcsam.DOI:10.6041/j.issn.1000-1298.2017.12.021.(in Chinese)
[19]YANG Lin,ZHAI Ruifang,SHI Pujuan,et al.Segmentation of crop organs through region growing in 3D space[C]∥International Conference on Agro-Geoinformatics,2016.
[20]CHETVERIKOV D,STEPANOV D,KRSEK P.Robust Euclidean alignment of 3D point sets:the trimmed iterative closest point algorithm[J].Image&Vision Computing,2005,23(3):299-309.
[2]FAN Zhenqi,SI Jingchun,HAN Xu,et al.Design mode for component-based virtual plant software[J].Journal Agricultural Science&Technology,2012,13(4):901-903.
[3]李诗锐,李琪,李海洋,等.基与Kinect V2的实时精确三维重建系统[J].软件学报,2016,27(10):2519-2529.LI Shirui,LI Qi,LI Haiyang,et al.Real-time accurate 3D reconstruction based on Kinect V2[J].Journal of Software,2016,27(10):2519-2529.(in Chinese)
[4]PAULUS S,SCHUMANN H,KUHLUMANN H,et al.High-precision laser scanning system for capturing 3D plant architecture and analysing growth of cereal plants[J].Biosystems Engineering,2014,121:1-11.
[5]YAO Yidan,LIN Chengda,ZHAI Ruifang,et al.Rapeseed 3D reconstruction and morphological parameter measurement based on laser point cloud[C]∥International Conference on Agro-Geoinformatics,2016.
[6]张伟洁,刘刚,郭彩玲,等.基于三维点云的苹果树叶片三维重建研究[J/OL].农业机械学报,2017,48(增刊):103-109.ZHANG Weijie,LIU Gang,GUO Cailing,et al.Apple tree leaf three-dimensional reconstruction based on point cloud[J/OL].Transactions of the Chinese Society for Agricultural Machinery,2017,48(Supp.):103-109.http:∥www.j-csam.org/jcsam/ch/reader/view_abstract.aspx?flag=1&file_no=2017s017&journal_id=jcsam.DOI:10.6041/j.issn.1000-1298.2017.S0.017.(in Chinese)
[7]XIONG Xiong,YU Lejun,YANG Wanneng,et al.A high-throughput stereo-imaging system for quantifying rape leaf traits during the seedling stage[J].Plant Methods,2017,13(1):7.
[8]王传宇,郭新宇,吴升,等.基于计算机视觉的玉米果穗三维重建方法[J/OL].农业机械学报,2014,45(9):274-279.WANG Chuanyu,GUO Xinyu,WU Sheng,et al.Three dimensional reconstruction of maize ear based on computer vision[J/OL].Transactions of the Chinese Society for Agricultural Machinery,2014,45(9):274-279.http:∥www.j-csam.org/jcsam/ch/reader/view_abstract.aspx?flag=1&file_no=20140944&journal_id=jcsam.DOI:10.6041/j.issn.1000-1298.2014.09.044.(in Chinese)
[9]张艳超,庄载椿,肖宇钊,等.基于运动恢复结构算法的油菜NDVI三维分布[J].农业工程学报,2015,31(17):207-214.ZHANG Yanchao,ZHUANG Zaichun,XIAO Yuzhao,et al.Rape plant NDVI 3D distribution based on structure from motion[J].Transactions of the CSAE,2015,31(17):207-214.(in Chinese)
[10]FANG Yihang,LIN Chengda,ZHAI Ruifang,et al.A new approach for measuring 3D digitalized rape leaf parameters based on images[C]∥International Conference on Agro-Geoinformatics,2014:1-5.
[11]宗泽,郭彩玲,张雪,等.基于深度相机的玉米株型参数提取方法研究[J/OL].农业机械学报,2015,46(增刊):50-56.ZONG Ze,GUO Cailing,ZHANG Xue,et al.Maize plant type parameters extraction based on depth camera[J/OL].Transactions of the Chinese Society for Agricultural Machinery,2015,46(Supp.):50-56.http:∥www.j-csam.org/jcsam/ch/reader/view_abstract.aspx?flag=1&file_no=2015S009&journal_id=jcsam.DOI:10.6041/j.issn.1000-1298.2015.S0.009.(in Chinese)
[12]SALAU J,HAAS J H,JUNGE W,et al.Automated calculation of udder depth and rear leg angle in Holstein-Friesian cows using a multi-Kinect cow scanning system[J].Biosystems Engineering,2017,160:154-169.
[13]麦春艳,郑立华,孙红,等.基于RGB-D相机的果树三维重构与果实识别定位[J/OL].农业机械学报,2015,46(增刊):35-40.MAI Chunyan,ZHENG Lihua,SUN Hong,et al.Research on 3D reconstruction of fruit tree and fruit recognition and location method based on RGB-D camera[J/OL].Transactions of the Chinese Society for Agricultural Machinery,2015,46(Supp.):35-40.http:∥www.j-csam.org/jcsam/ch/reader/view_abstract.aspx?flag=1&file_no=2015S006&journal_id=jcsam.DOI:10.6041/j.issn.1000-1298.2015.S0.006.(in Chinese)
[14]VITZRABIN E,EDAN Y.Adaptive thresholding with fusion using a RGB-D sensor for red sweet-pepper detection[J].Biosystems Engineering,2016,146:45-56.
[15]GUO Kaiwen,XU Feng,YU Tao,et al.Real-time geometry,albedo,and motion reconstruction using a single RGB-D camera[J].ACM Transactions on Graphics,2017,36(3):32.
[16]JUAN R,TERVEN D,MARGARITA C E,et al.A multiple camera calibration and point cloud fusion tool for Kinect V2[J].Science of Computer Programming,2017,143:1-8.
[17]郑立华,麦春艳,廖崴,等.基于Kinect相机的苹果树三维点云配准[J/OL].农业机械学报,2016,47(5):9-14.ZHENG Lihua,MAI Chunyan,LIAO Wei,et al.3D point cloud registration for apple tree based on Kinect camera[J/OL].Transactions of the Chinese Society for Agricultural Machinery,2016,47(5):9-14.http:∥www.j-csam.org/jcsam/ch/reader/view_abstract.aspx?flag=1&file_no=20160502&journal_id=jcsam.DOI:10.6041/j.issn.1000-1298.2016.05.002.(in Chinese)
[18]沈跃,潘成凯,刘慧,等.基于改进SIFT-ICP算法的Kinect植株点云配准方法[J/OL].农业机械学报,2017,48(12):183-189.SHEN Yue,PAN Chengkai,LIU Hui,et al.Method of plant point cloud registration based on Kinect of improved SIFT-ICP[J/OL].Transactions of the Chinese Society for Agricultural Machinery,2017,48(12):183-189.http:∥www.j-csam.org/jcsam/ch/reader/view_abstract.aspx?flag=1&file_no=20171221&journal_id=jcsam.DOI:10.6041/j.issn.1000-1298.2017.12.021.(in Chinese)
[19]YANG Lin,ZHAI Ruifang,SHI Pujuan,et al.Segmentation of crop organs through region growing in 3D space[C]∥International Conference on Agro-Geoinformatics,2016.
[20]CHETVERIKOV D,STEPANOV D,KRSEK P.Robust Euclidean alignment of 3D point sets:the trimmed iterative closest point algorithm[J].Image&Vision Computing,2005,23(3):299-309.