三维点云场景数据获取及其场景理解关键技术综述
详细信息    查看全文 | 推荐本文 |
  • 英文篇名:3D Point Cloud Scene Data Acquisition and Its Key Technologies for Scene Understanding
  • 作者:李勇 ; 佟国峰 ; 杨景超 ; 张立强 ; 彭浩 ; 高华帅
  • 英文作者:Li Yong;Tong Guofeng;Yang Jingchao;Zhang Liqiang;Peng Hao;Gao Huashuai;College of Information Science and Engineering,Northeastern University;Department of Electrical and Information Engineering,Hebei Jiaotong Vocational and Technical College;The State Key Laboratory of Remote Sensing Science,Beijing Normal University;
  • 关键词:机器视觉 ; 三维点云 ; 场景理解 ; 语义分割
  • 英文关键词:machine vision;;3D point cloud;;scene understanding;;semantic segmentation
  • 中文刊名:JGDJ
  • 英文刊名:Laser & Optoelectronics Progress
  • 机构:东北大学信息科学与工程学院;河北交通职业技术学院电气与信息工程系;北京师范大学遥感科学国家重点实验室;
  • 出版日期:2018-09-14 09:32
  • 出版单位:激光与光电子学进展
  • 年:2019
  • 期:v.56;No.639
  • 基金:国家自然科学基金(61175031);; 国家863计划(2012AA041402)
  • 语种:中文;
  • 页:JGDJ201904002
  • 页数:14
  • CN:04
  • ISSN:31-1690/TN
  • 分类号:21-34
摘要
场景理解是信息科学里的重要研究内容,而三维(3D)数据相比于二维(2D)数据有着众多优势。目前点云的获取有多种方式,且不同获取方式的点云具有不同的特点,此外,基于点云的3D场景理解中的关键技术研究还没有完整、系统的综述。为此,总结了不同方式的点云获取方法,并对不同的点云数据及相关数据库进行对比分析。基于目前3D场景理解的研究进展,针对3D场景理解中的点云滤波、特征提取与点云分割和点云语义分割等技术进行了对比分析与总结。通过对近些年国内外文献的结论进行梳理,凝练出3D场景理解关键技术中存在的问题,并对3D场景理解问题的发展趋势做了展望。基于点云的3D场景理解因其数据的丰富性而被广泛应用在众多领域中,但是目前基于3D点云的场景理解效果,尤其是针对具有颜色信息的激光点云的场景理解,还有众多内容有待深入研究。
        Scene understanding is an important research content in information science.Compared with the twodimensional(2 D)data,the three-dimensional(3D)data has many advantages.At present,there are many ways to acquire the point clouds,and meanwhile the point clouds with different acquisition methods have different characteristics.In addition,there lacks a complete and systematic research review on the key techniques for 3D scenes understanding.Thus,the different methods for point cloud acquisition are summarized,and the different point cloud data and related databases are compared and analyzed as well.Based on the current research progress of3 Dscene understanding,the techniques for point cloud filtering,feature extraction,point cloud segmentation,and point cloud semantic segmentation in 3 Dscene understanding are compared and summarized.By the summary of the domestic and foreign literatures published in recent years,the problems occurred in the key technologies for 3 D scene understanding are condensed,and the development trend of the 3 D scene understanding problems is prospected.The 3D scene understanding based on point clouds is widely used in many fields due to its richness of data.However,as for the scene understanding effect of 3D point clouds,especially the scene understanding of laser point clouds with color information,there are still many contents needed to be investigated in depth.
引文
[1]Kim B S,Kohli P,Savarese S.3D scene understanding by voxel-CRF[C]∥2013 IEEEInternational Conference on Computer Vision,December 1-8,2013,Sydney,NSW,Australia.New York:IEEE,2013:1425-1432.
    [2]Zhuang Y,He G J,Hu H S,et al.A novel outdoor scene-understanding framework for unmanned ground vehicles with 3Dlaser scanners[J].Transactions of the Institute of Measurement and Control,2015,37(4):435-445.
    [3]Tong G F,Du X C,Li Y,et al.Three-dimensional point cloud classification of large outdoor scenes based on vertical slice sampling and centroid distance histograms[J].Chinese Journal of Lasers,2018,45(10):1004001.佟国峰,杜宪策,李勇,等.基于切片采样和质心距直方图特征的室外大场景三维点云分类[J].中国激光,2018,45(10):1004001.
    [4]Yan F,Zhuang Y,Wang W.Outdoor scene comprehension of mobile robot based on multi-sensor information fusion[J].Control Theory&Applications,2011,28(8):1093-1098.闫飞,庄严,王伟.移动机器人基于多传感器信息融合的室外场景理解[J].控制理论与应用,2011,28(8):1093-1098.
    [5]Izadi S,Kim D,Hilliges O,et al.KinectFusion:real-time 3D reconstruction and interaction using a moving depth camera[C]∥Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology,October 16-19,2011.Santa Barbara,California,USA.New York:ACM,2011:559-568.
    [6]Lu X S,Yu J Y,Tian M Y,et al.Fusion method of vehicle laser point cloud and serialized panoramic image[J].Chinese Journal of Lasers,2018,45(5):0510004.卢秀山,俞家勇,田茂义,等.车载激光点云与序列化全景影像融合方法[J].中国激光,2018,45(5):0510004.
    [7]Bircher A,Alexis K,Burri M,et al.Structural inspection path planning via iterative viewpoint resampling with application to aerial robotics[C]∥IEEE International Conference on Robotics and Automation(ICRA),May 26-30,2015,Seattle,WA,USA.New York:IEEE,2015:6423-6430.
    [8]Hackel T,Savinov N,Ladicky L,et al.Semantic3D.net:a new large-scale point cloud classification benchmark[J].ISPRS Annals of Photogrammetry,Remote Sensing and Spatial Information Sciences,2017,4:91-98.
    [9]Landrieu L,Simonovsky M.Large-scale point cloud semantic segmentation with superpoint graphs[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR),June 19-21,2018,Salt Lake City,Utah,USA.New York:IEEE,2018:4558-4567.
    [10]Silberman N,Hoiem D,Kohli P,et al.Indoor segmentation and support inference from RGBDimages[C]∥European Conference on Computer Vision.Berlin,Heidelberg:Springer,2012:746-760.
    [11]Song S R,Lichtenberg S P,Xiao J X.SUN RGB-D:a RGB-D scene understanding benchmark suite[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR),June 7-12,2015,Boston,MA,USA.New York:IEEE,2015:567-576.
    [12]Chen C J,Wang G,Guan H L.Vehicle-mounted measure integrated system based on laser scanning and panorama images:CN202782968U[P].2013-03-13.陈长军,王刚,关鸿亮.基于激光扫描与全景影像的车载测量集成系统:CN202782968U[P].2013-03-13.
    [13]Felix R.Method of generating panorama views on a mobile mapping system:EP20140156863[P].2015-09-02.
    [14]Pomerleau F,Liu M,Colas F,et al.Challenging data sets for point cloud registration algorithms[J].The International Journal of Robotics Research,2012,31(14):1705-1711.
    [15]Sydney urban objects dataset[EB/OL](2013-11-04)[2018-07-18]http:∥www.acfr.usyd.edu.au/papers/SydneyUrbanObjectsDataset.shtml.
    [16]Wu Z,Song S,Khosla A,et al.3DshapeNets:Adeep representation for volumetric shapes[C]∥IEEEConference on Computer Vision and Pattern Recognition(CVPR),June 23-28,2014,Columbus,OH,USA.New York:IEEE,2014:1912-1920.
    [17]Johnson A E.Spin-images:a representation for 3-Dsurface matching[D].Pennsylvania:Carnegie Mellon University,1997.
    [18]Frome A,Huber D,Kolluri R,et al.Recognizing objects in range data using regional point descriptors[C]∥European Conference on Computer Vision.Berlin,Heidelberg:Springer,2004:224-237.
    [19]Rusu R B,Blodow N,Marton Z C,et al.Aligning point cloud views using persistent feature histograms[C]∥IEEE/RSJ International Conference on Intelligent Robots and Systems,September 22-26,2008,Nice,France.New York:IEEE,2008:3384-3391.
    [20]Salti S,Tombari F,Di Stefano L.SHOT:unique signatures of histograms for surface and texture description[J].Computer Vision and Image Understanding,2014,125:251-264.
    [21]Wohlkinger W,Vincze M.Ensemble of shape functions for 3D object classification[C]∥IEEEInternational Conference on Robotics and Biomimetics,December 7-11,2011,Karon Beach,Phuket,Thailand.New York:IEEE,2011:2987-2992.
    [22]Rusu R B,Bradski G,Thibaux R,et al.Fast 3Drecognition and pose using the viewpoint feature histogram[C]∥IEEE/RSJ International Conference on Intelligent Robots and Systems,October 18-22,2010,Taipei,Taiwan,China.New York:IEEE,2010:2155-2162.
    [23]Rusu R B,Blodow N,Beetz M.Fast point feature histograms(FPFH)for 3Dregistration[C]∥IEEEInternational Conference on Robotics and Automation,May 12-17,2009,Kobe,Japan,New York:IEEE,2009:3212-3217.
    [24]Sappa A D,Devy M.Fast range image segmentation by an edge detection strategy[C]∥Proceedings Third International Conference on 3-D Digital Imaging and Modeling,May 28-June 1,2001,Quebec City,Quebec,Canada.New York:IEEE,2001:292-299.
    [25]Ke Y L,Shan D R.Edge-based segmentation of point cloud data[J].Journal of Zhejiang University(Engineering Science),2005,39(3):377-380.柯映林,单东日.基于边特征的点云数据区域分割[J].浙江大学学报(工学版),2005,39(3):377-380.
    [26]Papon J,Abramov A,Schoeler M,et al.Voxel cloud connectivity segmentation-supervoxels for point clouds[C]∥IEEE Conference on Computer Vision and Pattern Recognition,June 23-28,2013,Portland,OR,USA.New York:IEEE,2013:2027-2034.
    [27]Vo A V,Truong-Hong L,Laefer D F,et al.Octreebased region growing for point cloud segmentation[J].ISPRS Journal of Photogrammetry and Remote Sensing,2015,104:88-100.
    [28]Yang Z X,Cheng X J,Li Q,et al.Segmentation of point cloud in tank of plane bulkhead type[J].Chinese Journal of Lasers,2017,44(10):1010006.杨泽鑫,程效军,李泉,等.平面舱壁类型的船舱点云分割方法[J].中国激光,2017,44(10):1010006.
    [29]Wang Y M,Shi H B.A segmentation method for point cloud based on local sample and statistic inference[M]∥Bian F L,Xie Y C.eds.GeoInformatics in Resource Management and Sustainable Ecosystem.Berlin,Heidelberg:Springer,2015:274-282.
    [30]Green W R,Grobler H.Normal distribution transform graph-based point cloud segmentation[C]∥Pattern Recognition Association of South Africa and Robotics and Mechatronics International Conference,November 26-27,2015,Port Elizabeth,South Africa.New York:IEEE,2015:54-59.
    [31]Yang J Y,Gan Z Q,Li K,et al.Graph-based segmentation for RGB-D data using 3-D geometry enhanced superpixels[J].IEEE Transactions on Cybernetics,2015,45(5):913-926.
    [32]Cheng X J,Cheng X L,Hu M J,et al.Buildings detection and contour extraction by fusion of aerial images and LIDAR point cloud[J].Chinese Journal of Lasers,2016,43(5):0514002.程效军,程小龙,胡敏捷,等.融合航空影像和LIDAR点云的建筑物探测及轮廓提取[J].中国激光,2016,43(5):0514002.
    [33]Wolf D,Prankl J,Vincze M.Fast semantic segmentation of 3Dpoint clouds using a dense CRFwith learned parameters[C]∥IEEE International Conference on Robotics and Automation(ICRA),May 26-30,2015,Seattle,WA,USA.New York:IEEE,2015:4867-4873.
    [34]Hackel T,Wegner J D,Schindler K.Fast semantic segmentation of 3Dpoint clouds with strongly varying density[J].ISPRS Annals of Photogrammetry,Remote Sensing and Spatial Information Sciences,2016,3(3):177-184.
    [35]Weinmann M,Jutzi B,Hinz S,et al.Semantic point cloud interpretation based on optimal neighborhoods,relevant features and efficient classifiers[J].ISPRSJournal of Photogrammetry and Remote Sensing,2015,105:286-304.
    [36]Xiang B B,Yao J,Lu X H,et al.Segmentationbased classification for 3Durban point clouds[C]∥IEEE International Conference on Information and Automation(ICIA),August 1-3,2016,Ningbo,China.New York:IEEE,2016:172-177.
    [37]Aijazi A K,Checchin P,Trassoudaine L.Segmentation based classification of 3D urban point clouds:a super-voxel based approach with evaluation[J].Remote Sensing,2013,5(4):1624-1650.
    [38]Ghamisi P,H9fle B.LiDAR data classification using extinction profiles and a composite kernel support vector machine[J].IEEE Geoscience and Remote Sensing Letters,2017,14(5):659-663.
    [39]Lodha S K,Fitzpatrick D M,Helmbold D P.Aerial lidar data classification using AdaBoost[C]∥Sixth International Conference on 3-D Digital Imaging and Modeling(3DIM 2007),August 21-23,2007,Montreal,QC,Canada.New York:IEEE,2007:435-442.
    [40]Ni H,Lin X G,Zhang J X.Classification of ALSpoint cloud with improved point cloud segmentation and random forests[J].Remote Sensing,2017,9(3):288.
    [41]Niemeyer J,Rottensteiner F,Soergel U.Classification of urban LiDAR data using conditional random field and random forests[C]∥Joint Urban Remote Sensing Event 2013,April 21-23,2013,Sao Paulo,Brazil.New York:IEEE,2013:139-142.
    [42]Su H,Maji S,Kalogerakis E,et al.Multi-view convolutional neural networks for 3D shape recognition[C]∥IEEE International Conference on Computer Vision(ICCV),December 7-13,2015,Santiago,Chile.New York:IEEE,2015:945-953.
    [43]Qi C R,Su H,Niener M,et al.Volumetric and multi-view CNNs for object classification on 3Ddata[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR),June 27-30,2016,Las Vegas,NV,USA.New York:IEEE,2016:5648-5656.
    [44]Boulch A,Guerry J,le Saux B,et al.SnapNet:3Dpoint cloud semantic labeling with 2D deep segmentation networks[J].Computers&Graphics,2018,71:189-198.
    [45]Lawin F J,Danelljan M,Tosteberg P,et al.Deep projective 3D semantic segmentation[C]∥International Conference on Computer Analysis of Images and Patterns.Cham:Springer,2017:95-107.
    [46]Hackel T,Savinov N,Ladicky L,et al.SEMANTIC3D.NET:a new large-scale point cloud classification benchmark[M/OL].ISPRS annals of the photogrammetry,remote sensing and spatial information sciences.2017,IV-1-W1:91-98.[2018-07-20].http:∥www.semantic3d.net/view_results.php?chl=2.
    [47]Maturana D and Sebastian S.VoxNet:a 3Dconvolutional neural network for real-time object recognition[C]∥IEEE/RSJ International Conference on Intelligent Robots and Systems,Congress Center Hamburg,September 28-October 2,2015,Hamburg,Germany.New York:IEEE,2015:922-928.
    [48]Riegler G,Ulusoy A O,Geiger A.OctNet:learning deep 3D representations at high resolutions[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR),July 21-26,2017,Honolulu,HI,USA.New York:IEEE,2017,3:6620-6629.
    [49]Huang J,You S Y.Point cloud labeling using 3Dconvolutional neural network[C]∥23rd International Conference on Pattern Recognition(ICPR),December 4-8,2016,Cancun,Mexico.New York:IEEE,2016:2670-2675.
    [50]Roynard X,Deschaud J E,Fran9ois G.Classification of point cloud scenes with multiscale voxel deep network[EB/OL].(2018-04-10)[2018-07-18].https:∥arxiv.org/abs/1804.03583.
    [51]Wang L,Huang Y C,Shan J,et al.MSNet:multiscale convolutional network for point cloud classification[J].Remote Sensing,2018,10(4):612.
    [52]Tchapmi L,Choy C,Armeni I,et al.SEGCloud:semantic segmentation of 3D point clouds[C]∥International Conference on 3D Vision(3DV),October 10-12,2017,Qingdao,China.New York:IEEE,2017:537-547.
    [53]Charles R Q,Su H,Mo K C,et al.PointNet:deep learning on point sets for 3D classification and segmentation[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR),July 21-26,2017,Honolulu,HI,USA.New York:IEEE,2017:77-85.
    [54]Li Y Y,Bu R,Sun M C,et al.PointCNN:convolution onχ-transformed points[EB/OL].(2018-11-05)[2018-07-15].https:∥arxiv.org/abs/1801.07791
    [55]Li J,Chen B M,Lee G H.SO-net:self-organizing network for point cloud analysis[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,June 18-23,2018.Salt Lake City,UT,USA.New York:IEEE,2018:9397-9406.
    [56]Behl A,Paschalidou D,DonnéS,et al.PointFlowNet:learning representations for 3Dscene flow estimation from point clouds[EB/OL].(2018-06-06)[2018-07-18].https:∥arxiv.org/abs/1806.02170.

© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号

地址:北京市海淀区学院路29号 邮编:100083

电话:办公室:(+86 10)66554848;文献借阅、咨询服务、科技查新:66554700