结合全局与局部信息的点云目标识别模型库构建
|
摘要
目的点云目标识别流程分为离线与在线阶段。离线阶段基于待识别目标的CAD模型构建一个模型库,在线基于近邻查找完成识别。本文针对离线阶段,提出一种新的模型库构建方法。方法首先将CAD模型置于一个二十面体中心,使用多个虚拟相机获取CAD模型在不同视角下的点云;然后将每个不同视角下的点云进行主成分分析并基于主成分分析的结果从多个选定的方向将点云切分为多个子部分,这些子部分包含点云的全局及局部信息;接着对每个子部分使用聚类算法获取其最大聚类,去除离群点;最后结合多种方式删减一些冗余聚类,减小模型库规模。结果在多个公开数据集上使用多种点云描述子进行对比实验,识别结果表明,相对于传统的模型库构建方法,基于本文方法进行识别正确率更高,在某些点云描述子上的识别正确率提升达到10%以上。结论通过将CAD模型在不同视角下点云的全局与局部信息都加入模型库中,本文提出的模型库构建方法可有效提高点云目标识别正确率,改善了场景目标发生遮挡时,近邻查找识别精度不高的问题。
Objective Frameworks for point cloud object recognition are generally composed by two stages. An offline stageconstructs a model library,and an online stage recognizes objects by using nearest neighbor search. Traditional methods useglobal surfaces to construct a model library,which is sensitive to occlusion and inaccurate segmentation result. This studyinvestigates the offline stage and presents a novel model library construction method. Method The proposed method simu-lates possible occlusions and adds point clouds with simulated occlusions to the model library to alleviate the influence ofocclusion and inaccurate segmentation result. First,a CAD model is placed at the center of an icosahedron,and multiplevirtual cameras are used to obtain the partial point clouds of the model. For each partial point cloud,a local coordinate sys-tem is constructed using principal component analysis,and the point cloud is aligned with the coordinate system. Thisprocess makes the proposed method invariant to rigid transformations. Second,several direction vectors are obtained basedon the local coordinate system,and the partial point clouds are segmented into multiple subparts based on the length of thepoint cloud on each direction vector. Simulation of occlusion at different degrees is performed on these subparts,which con-tain the global and local surfaces of the partial point cloud. Third,a simple clustering method is used to obtain the largestcluster of the subparts,and outliner points are removed at this stage. The largest cluster will be added to the model library only if the cluster has sufficient points. This process reduces the memory requirements and decreases time consumption during the nearest searches. Redundant clusters with similar surface in the library are still observed after removing the clusters with few points. Finally,an iterative closest point( ICP) based algorithm is used to remove the point clouds with similar surfaces,thereby further decreasing the memory requirements. Subsequently,only dozens of subparts are used to describe each of the CAD model. Result Experimental results on two public datasets show that the proposed method promotes recognition accuracy at different levels. For the UWAOR dataset,the recognition performance on five types of point cloud descriptor is remarkably improved. Particularly,the proposed method enhances the recognition performance by 0. 208 on the GASD descriptor and 0. 173 on the ROPS descriptor( k = 1 in KNN). For the Bologna Random Views dataset,the proposed method enhances the recognition accuracy of most of the point cloud descriptors. For example,the proposed method improves the recognition rate by 0. 193( k = 1) for the GASD descriptor. However,the recognition improvement on Bologna Random Views dataset is slightly lower than that of the UWAOR dataset. This condition is partially caused by the lighter occlusion of scene objects on Bologna Random Views dataset compared with the UWAOR dataset. Experiments at different noise levels are also conducted. The noise that follows a Gaussian distribution with different variances and zero means are added to the scene point cloud. Experimental results show that the proposed method maintains the recognition rate promotion with the increase on the standard deviation of noise. For example,the proposed method enhances the recognition rate of ESF descriptor by 0. 162( no noise) and 0. 034( noise with a standard deviation of 3 × mesh resolution) for the UWAOR dataset. This finding can be interpreted as the subparts having considerable points to overcome the influence of noise. Conclusion The proposed method enhances the recognition performance by combining the global and local surfaces of partial point clouds in constructing the model library,especially when the scene objects are occluded or have inaccurate segmentation. This outcome is valuable because it reduces the time consumption of the subsequent hypothesis verification stage. A better redundancy reduction algorithm should be proposed in future studies,in which each of the CAD model can be represented with the same number of subparts. In the present work,various subparts are used to describe different CAD models,which has affected the recognition results of nearest neighbor search. Meanwhile,the coarse pose of the scene object can be estimated by aligning the scene object with the point cloud in the model library,and the ICP-based algorithm can be used to refine the coarse pose to obtain precise pose information.
Objective Frameworks for point cloud object recognition are generally composed by two stages. An offline stageconstructs a model library,and an online stage recognizes objects by using nearest neighbor search. Traditional methods useglobal surfaces to construct a model library,which is sensitive to occlusion and inaccurate segmentation result. This studyinvestigates the offline stage and presents a novel model library construction method. Method The proposed method simu-lates possible occlusions and adds point clouds with simulated occlusions to the model library to alleviate the influence ofocclusion and inaccurate segmentation result. First,a CAD model is placed at the center of an icosahedron,and multiplevirtual cameras are used to obtain the partial point clouds of the model. For each partial point cloud,a local coordinate sys-tem is constructed using principal component analysis,and the point cloud is aligned with the coordinate system. Thisprocess makes the proposed method invariant to rigid transformations. Second,several direction vectors are obtained basedon the local coordinate system,and the partial point clouds are segmented into multiple subparts based on the length of thepoint cloud on each direction vector. Simulation of occlusion at different degrees is performed on these subparts,which con-tain the global and local surfaces of the partial point cloud. Third,a simple clustering method is used to obtain the largestcluster of the subparts,and outliner points are removed at this stage. The largest cluster will be added to the model library only if the cluster has sufficient points. This process reduces the memory requirements and decreases time consumption during the nearest searches. Redundant clusters with similar surface in the library are still observed after removing the clusters with few points. Finally,an iterative closest point( ICP) based algorithm is used to remove the point clouds with similar surfaces,thereby further decreasing the memory requirements. Subsequently,only dozens of subparts are used to describe each of the CAD model. Result Experimental results on two public datasets show that the proposed method promotes recognition accuracy at different levels. For the UWAOR dataset,the recognition performance on five types of point cloud descriptor is remarkably improved. Particularly,the proposed method enhances the recognition performance by 0. 208 on the GASD descriptor and 0. 173 on the ROPS descriptor( k = 1 in KNN). For the Bologna Random Views dataset,the proposed method enhances the recognition accuracy of most of the point cloud descriptors. For example,the proposed method improves the recognition rate by 0. 193( k = 1) for the GASD descriptor. However,the recognition improvement on Bologna Random Views dataset is slightly lower than that of the UWAOR dataset. This condition is partially caused by the lighter occlusion of scene objects on Bologna Random Views dataset compared with the UWAOR dataset. Experiments at different noise levels are also conducted. The noise that follows a Gaussian distribution with different variances and zero means are added to the scene point cloud. Experimental results show that the proposed method maintains the recognition rate promotion with the increase on the standard deviation of noise. For example,the proposed method enhances the recognition rate of ESF descriptor by 0. 162( no noise) and 0. 034( noise with a standard deviation of 3 × mesh resolution) for the UWAOR dataset. This finding can be interpreted as the subparts having considerable points to overcome the influence of noise. Conclusion The proposed method enhances the recognition performance by combining the global and local surfaces of partial point clouds in constructing the model library,especially when the scene objects are occluded or have inaccurate segmentation. This outcome is valuable because it reduces the time consumption of the subsequent hypothesis verification stage. A better redundancy reduction algorithm should be proposed in future studies,in which each of the CAD model can be represented with the same number of subparts. In the present work,various subparts are used to describe different CAD models,which has affected the recognition results of nearest neighbor search. Meanwhile,the coarse pose of the scene object can be estimated by aligning the scene object with the point cloud in the model library,and the ICP-based algorithm can be used to refine the coarse pose to obtain precise pose information.
引文
[1]Guo Y L,Bennamoun M,Sohel F A,et al.An integrated framework for 3-D modeling,object detection,and pose estimation from point-clouds[J].IEEE Transactions on Instrumentation and Measurement,2015,64(3):683-693.[DOI:10.1109/TIM.2014.2358131]
[2]Li Q,Zhou M L,Liu J.A review on 3D objects recognition[J].Journal of Image and Graphics,2000,5(12):985-993.[李庆,周曼丽,柳健.3维物体识别研究进展[J].中国图象图形学报,2000,5(12):985-993.][DOI:10.11834/jig.20001202]
[3]do Monte Lima J P S,Teichrieb V.An efficient global point cloud descriptor for object recognition and pose estimation[C]//Proceedings of the 29th SIBGRAPI Conference on Graphics,Patterns and Images.Sao Paulo,Brazil:IEEE,2016:56-63.[DOI:10.1109/SIBGRAPI.2016.017]
[4]Wang Y M,Pan G,Wu Z H.A survey of 3D face recognition[J].Journal of Computer-Aided Design&Computer Graphics,2008,20(7):819-829.[王跃明,潘纲,吴朝晖.3维人脸识别研究综述[J].计算机辅助设计与图形学学报,2008,20(7):819-829.]
[5]Szegedy C,Vanhoucke V,Ioffe S,et al.Rethinking the inception architecture for computer vision[C]//Proceedings of 2016IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,NV,USA:IEEE,2016:2818-2826.[DOI:10.1109/CVPR.2016.308]
[6]Redmon J,Divvala S K,Girshick R B,et al.You only look once:unified,real-time object detection[C]//Proceedings of2016 IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,NV,USA:IEEE,2016:779-788.[DOI:10.1109/CVPR.2016.91]
[7]Ioannidou A,Chatzilari E,Nikolopoulos S,et al.Deep learning advances in computer vision with 3D data:a survey[J].ACMComputing Surveys,2017,50(2):20.[DOI:10.1145/3042064]
[8]Rusu R B,Bradski G,Thibaux R,et al.Fast 3D recognition and pose using the viewpoint feature histogram[C]//Proceedings of 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.Taipei,Taiwan:IEEE,2010:2155-2162.[DOI:10.1109/IROS.2010.5651280]
[9]Wohlkinger W,Vincze M.Ensemble of shape functions for 3Dobject classification[C]//Proceedings of 2011 IEEE International Conference on Robotics and Biomimetics.Karon Beach,Thailand:IEEE,2011:2987-2992.[DOI:10.1109/ROBIO.2011.6181760]
[10]Rusu R B,Blodow N,Marton Z C,et al.Aligning point cloud views using persistent feature histograms[C]//Proceedings of2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.Nice,France:IEEE,2008:3384-3391.[DOI:10.1109/IROS.2008.4650967]
[11]Rusu R B,Blodow N,Beetz M.Fast point feature histograms(FPFH)for 3D registration[C]//Proceedings of 2009 IEEE International Conference on Robotics and Automation.Kobe,Japan:IEEE,2009:3212-3217.[DOI:10.1109/ROBOT.2009.5152473]
[12]Tombari F,Salti S,Di Stefano L.Unique signatures of histograms for local surface description[C]//Proceedings of 2010 European Conference on Computer Vision.Crete,Greece:Springer,2010:356-369.[DOI:10.1007/978-3-642-15558-1_26]
[13]Guo Y L,Sohel F,Bennamoun M,et al.Rotational projection statistics for 3D local surface description and object recognition[J]International Journal of Computer Vision,2013,105(1):63-86.[DOI:10.1007/s11263-013-0627-y]
[14]Darom T,Keller Y.Scale-invariant features for 3-D mesh models[J].IEEE Transactions on Image Processing,2012,21(5):2758-2769.[DOI:10.1109/TIP.2012.2183142]
[15]Toldo R,Castellani U,Fusiello A.A bag of words approach for3D object categorization[C]//Proceedings of 2009 International Conference on Computer Vision/Computer Graphics Collaboration Techniques and Applications.Rocquencourt,France:Springer,2009:116-127.[DOI:10.1007/978-3-642-01811-4_11]
[16]Lei H,Jiang G,Quan L.Fast descriptors and correspondence propagation for robust global point cloud registration[J].IEEETransactions on Image Processing,2017,26(8):3614-3623.[DOI:10.1109/TIP.2017.2700727]
[17]Besl P J,Mc Kay H D.A method for registration of 3-D shapes[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1992,14(2):239-256.[DOI:10.1109/34.121791]
[18]Rusu R B,Cousins S.3D is here:Point cloud library(PCL)[C]//Proceedings of 2011 IEEE International Conference on Robotics and Automation.Shanghai,China:IEEE,2011:1-4.[DOI:10.1109/ICRA.2011.5980567]
[19]Zhang S M,Wu J P,Zhou K S.Spherical triangle subdivision and analysis based on polyhedron[J].Computer Engineering and Applications,2008,44(9):16-19.[张胜茂,吴健平,周科松.基于正多面体的球面三角剖分与分析[J].计算机工程与应用,2008,44(9):16-19.][DOI:10.3778/j.issn.1002-8331.2008.09.005]
[20]Rabbani T,van den Heuvel F,Vosselmann G.Segmentation of point clouds using smoothness constraint[J].International Society for Photogrammetry and Remote Sensing,2006,35(6):248-253.
[21]Mian A S,Bennamoun M,Owens R.Three-dimensional modelbased object recognition and segmentation in cluttered scenes[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2006,28(10):1584-1601.[DOI:10.1109/TPAMI.2006.213]
[22]Mian A,Bennamoun M,Owens R.On the repeatability and quality of keypoints for local feature-based 3D object retrieval from cluttered scenes[J].International Journal of Computer Vision,2010,89(2-3):348-361.[DOI:10.1007/s11263-009-0296-z]
[23]Muja M,Lowe D G.Fast approximate nearest neighbors with automatic algorithm configuration[C]//Proceedings of the International Conference on Computer Vision Theory and Application.Lisbon,Portugal:Sci Te Press,2009:331-340.[DOI:10.5220/0001787803310340]
[2]Li Q,Zhou M L,Liu J.A review on 3D objects recognition[J].Journal of Image and Graphics,2000,5(12):985-993.[李庆,周曼丽,柳健.3维物体识别研究进展[J].中国图象图形学报,2000,5(12):985-993.][DOI:10.11834/jig.20001202]
[3]do Monte Lima J P S,Teichrieb V.An efficient global point cloud descriptor for object recognition and pose estimation[C]//Proceedings of the 29th SIBGRAPI Conference on Graphics,Patterns and Images.Sao Paulo,Brazil:IEEE,2016:56-63.[DOI:10.1109/SIBGRAPI.2016.017]
[4]Wang Y M,Pan G,Wu Z H.A survey of 3D face recognition[J].Journal of Computer-Aided Design&Computer Graphics,2008,20(7):819-829.[王跃明,潘纲,吴朝晖.3维人脸识别研究综述[J].计算机辅助设计与图形学学报,2008,20(7):819-829.]
[5]Szegedy C,Vanhoucke V,Ioffe S,et al.Rethinking the inception architecture for computer vision[C]//Proceedings of 2016IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,NV,USA:IEEE,2016:2818-2826.[DOI:10.1109/CVPR.2016.308]
[6]Redmon J,Divvala S K,Girshick R B,et al.You only look once:unified,real-time object detection[C]//Proceedings of2016 IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,NV,USA:IEEE,2016:779-788.[DOI:10.1109/CVPR.2016.91]
[7]Ioannidou A,Chatzilari E,Nikolopoulos S,et al.Deep learning advances in computer vision with 3D data:a survey[J].ACMComputing Surveys,2017,50(2):20.[DOI:10.1145/3042064]
[8]Rusu R B,Bradski G,Thibaux R,et al.Fast 3D recognition and pose using the viewpoint feature histogram[C]//Proceedings of 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.Taipei,Taiwan:IEEE,2010:2155-2162.[DOI:10.1109/IROS.2010.5651280]
[9]Wohlkinger W,Vincze M.Ensemble of shape functions for 3Dobject classification[C]//Proceedings of 2011 IEEE International Conference on Robotics and Biomimetics.Karon Beach,Thailand:IEEE,2011:2987-2992.[DOI:10.1109/ROBIO.2011.6181760]
[10]Rusu R B,Blodow N,Marton Z C,et al.Aligning point cloud views using persistent feature histograms[C]//Proceedings of2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.Nice,France:IEEE,2008:3384-3391.[DOI:10.1109/IROS.2008.4650967]
[11]Rusu R B,Blodow N,Beetz M.Fast point feature histograms(FPFH)for 3D registration[C]//Proceedings of 2009 IEEE International Conference on Robotics and Automation.Kobe,Japan:IEEE,2009:3212-3217.[DOI:10.1109/ROBOT.2009.5152473]
[12]Tombari F,Salti S,Di Stefano L.Unique signatures of histograms for local surface description[C]//Proceedings of 2010 European Conference on Computer Vision.Crete,Greece:Springer,2010:356-369.[DOI:10.1007/978-3-642-15558-1_26]
[13]Guo Y L,Sohel F,Bennamoun M,et al.Rotational projection statistics for 3D local surface description and object recognition[J]International Journal of Computer Vision,2013,105(1):63-86.[DOI:10.1007/s11263-013-0627-y]
[14]Darom T,Keller Y.Scale-invariant features for 3-D mesh models[J].IEEE Transactions on Image Processing,2012,21(5):2758-2769.[DOI:10.1109/TIP.2012.2183142]
[15]Toldo R,Castellani U,Fusiello A.A bag of words approach for3D object categorization[C]//Proceedings of 2009 International Conference on Computer Vision/Computer Graphics Collaboration Techniques and Applications.Rocquencourt,France:Springer,2009:116-127.[DOI:10.1007/978-3-642-01811-4_11]
[16]Lei H,Jiang G,Quan L.Fast descriptors and correspondence propagation for robust global point cloud registration[J].IEEETransactions on Image Processing,2017,26(8):3614-3623.[DOI:10.1109/TIP.2017.2700727]
[17]Besl P J,Mc Kay H D.A method for registration of 3-D shapes[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1992,14(2):239-256.[DOI:10.1109/34.121791]
[18]Rusu R B,Cousins S.3D is here:Point cloud library(PCL)[C]//Proceedings of 2011 IEEE International Conference on Robotics and Automation.Shanghai,China:IEEE,2011:1-4.[DOI:10.1109/ICRA.2011.5980567]
[19]Zhang S M,Wu J P,Zhou K S.Spherical triangle subdivision and analysis based on polyhedron[J].Computer Engineering and Applications,2008,44(9):16-19.[张胜茂,吴健平,周科松.基于正多面体的球面三角剖分与分析[J].计算机工程与应用,2008,44(9):16-19.][DOI:10.3778/j.issn.1002-8331.2008.09.005]
[20]Rabbani T,van den Heuvel F,Vosselmann G.Segmentation of point clouds using smoothness constraint[J].International Society for Photogrammetry and Remote Sensing,2006,35(6):248-253.
[21]Mian A S,Bennamoun M,Owens R.Three-dimensional modelbased object recognition and segmentation in cluttered scenes[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2006,28(10):1584-1601.[DOI:10.1109/TPAMI.2006.213]
[22]Mian A,Bennamoun M,Owens R.On the repeatability and quality of keypoints for local feature-based 3D object retrieval from cluttered scenes[J].International Journal of Computer Vision,2010,89(2-3):348-361.[DOI:10.1007/s11263-009-0296-z]
[23]Muja M,Lowe D G.Fast approximate nearest neighbors with automatic algorithm configuration[C]//Proceedings of the International Conference on Computer Vision Theory and Application.Lisbon,Portugal:Sci Te Press,2009:331-340.[DOI:10.5220/0001787803310340]