基于智能手机Tango传感器的室内三维场景实时感知
|
摘要
室内三维场景实时感知是室内增强现实、三维测图和机器人自主定位与导航的关键技术之一。利用智能手机Tango传感器搭载的单目摄像头和深度相机,根据实时获取的彩色图像和深度信息,研究并提出了一种实时的室内三维场景感知方法;该方法首先根据智能手机Tango传感器获取的彩色图像存在的问题,采用多项式函数对彩色相机进行校正。接着,使用ORB(oriented FAST and rotated BRIEF)算子对彩色图像进行特征提取,利用RANSAC算法剔除误匹配点;并根据优化后的匹配点对采用ICP算法对手机姿态进行初步估计。最后,基于融合空间关系的视觉词典(SDBo W2)对感知的初步室内场景进行闭环检测;并通过图优化得到实时感知的室内三维场景模型。实验表明,方法能利用Tango传感器高效地获取室内场景完整的三维结构信息与纹理。
Real-time perception of indoor 3D scene is one of the key technologies of indoor augmented reality,robot autonomous navigation and 3 D mapping. The approach of real-time perception of indoor 3D scene integrating real-time RGB image and depth map acquired from monocular camera and RGB-D camera in the Tango smart phone was proposed. Firstly,for the noise affection of RGB image and depth map from the Tango smart phone,the polynomial function was utilized to calibrate camera and Gaussian filter to denoise. Then,ORB( oriented FAST and rotated BRIEF) algorithm was used to extract image feature and delete outlines via the RANSAC algorithm,and the ICP algorithm was utilized to initially estimate the phone pose based on optimizing matching pairs. Finally,loop closure detection was used in initial perception indoor scene based on bags of binary words with spatial information( SDBoW2),and the real-time perception of indoor 3D scene model after graph optimization was accomplish experiment demonstrates that the approach proposed could acquire the whole 3D structure and texture of indoor scene effectively.
Real-time perception of indoor 3D scene is one of the key technologies of indoor augmented reality,robot autonomous navigation and 3 D mapping. The approach of real-time perception of indoor 3D scene integrating real-time RGB image and depth map acquired from monocular camera and RGB-D camera in the Tango smart phone was proposed. Firstly,for the noise affection of RGB image and depth map from the Tango smart phone,the polynomial function was utilized to calibrate camera and Gaussian filter to denoise. Then,ORB( oriented FAST and rotated BRIEF) algorithm was used to extract image feature and delete outlines via the RANSAC algorithm,and the ICP algorithm was utilized to initially estimate the phone pose based on optimizing matching pairs. Finally,loop closure detection was used in initial perception indoor scene based on bags of binary words with spatial information( SDBoW2),and the real-time perception of indoor 3D scene model after graph optimization was accomplish experiment demonstrates that the approach proposed could acquire the whole 3D structure and texture of indoor scene effectively.
引文
1 Zhang G,Jin H L,Lim J,et al.Building a 3-D line-based map using stereo SLAM.IEEE Transactions on Robotics,2015;31(6):1364-1377
2 Xu G,Yuan J,Li X,et al.3D reconstruction of laser projective point with projection invariant generated from five points on 2D target.Scientific Reports,2017;7(6):601-613
3 Bleier M,Nüchter A.Low-cost 3D laser scanning in air or water using self-calibrating structured light low-cost 3D laser scanning in air or water using self-calibrating structured light.3D ARCH 2017.Napflio:3D ARCH,2017:105-112
4向云飞,余代俊,张兵,等.基于Lidar数据和倾斜摄影的城市三维模型构建.测绘工程,2016;25(12):65-69Xiang Yunfei,Yu Daijun,Zhang Bing,et al.The construction of 3Dcity model based on lidar data and tilt photography.Engineering of Surveying and Mapping,2016;25(12):65-69
5张玉龙,张国山.基于关键帧的视觉惯性SLAM闭环检测算法.计算机科学与探索,2018.http://kns.cnki.net/kcms/detail/11.5602.TP.20180410.1649.002.html Zhang Yulong,Zhang Guoshan.Loop-closing detection algorithm of keyframe-based visual-inertial SLAM.Journal of Frontiers of Computer Science and Technology,2018.http://kns.cnki.net/kcms/detail/11.5602.TP.20180410.1649.002.html
6 Rublee E,Rabaud V,Konolige K,et al.ORB:An efficient alternative to SIFT or SURF.ICCV'11 Proceedings of the 2011 International Conference on Computer Vision.Washington:IEEE Computer Society,2012:2564-2571
7 Mur-Artal R,Montiel J M M,Tardós J D.ORB-SLAM:A versatile and accurate monocular SLAM system.IEEE Transactions on Robotics,2015;31(5):1147-1163
8 Khoshelham K,Elberink S O.Accuracy and resolution of Kinect depth data for indoor mapping applications.Sensors,2012;12(2):1437-1457
9 Dai A,Izadi S,Theobalt C.Bundle fusion:Real-time globally consistent 3D reconstruction using on-the-fly surface re-integration.ACMTransactions on Graphics,2017;36(4):1-17
10陈之杰.基于RGB-D感知的移动机器人三维地图构建与导航研究及应用.南京:东南大学,2016Chen Zhijie.Research and application of mobile robot 3d mapping and navigation with RGB-D perception.Nanjing:Southeast University,2016
2 Xu G,Yuan J,Li X,et al.3D reconstruction of laser projective point with projection invariant generated from five points on 2D target.Scientific Reports,2017;7(6):601-613
3 Bleier M,Nüchter A.Low-cost 3D laser scanning in air or water using self-calibrating structured light low-cost 3D laser scanning in air or water using self-calibrating structured light.3D ARCH 2017.Napflio:3D ARCH,2017:105-112
4向云飞,余代俊,张兵,等.基于Lidar数据和倾斜摄影的城市三维模型构建.测绘工程,2016;25(12):65-69Xiang Yunfei,Yu Daijun,Zhang Bing,et al.The construction of 3Dcity model based on lidar data and tilt photography.Engineering of Surveying and Mapping,2016;25(12):65-69
5张玉龙,张国山.基于关键帧的视觉惯性SLAM闭环检测算法.计算机科学与探索,2018.http://kns.cnki.net/kcms/detail/11.5602.TP.20180410.1649.002.html Zhang Yulong,Zhang Guoshan.Loop-closing detection algorithm of keyframe-based visual-inertial SLAM.Journal of Frontiers of Computer Science and Technology,2018.http://kns.cnki.net/kcms/detail/11.5602.TP.20180410.1649.002.html
6 Rublee E,Rabaud V,Konolige K,et al.ORB:An efficient alternative to SIFT or SURF.ICCV'11 Proceedings of the 2011 International Conference on Computer Vision.Washington:IEEE Computer Society,2012:2564-2571
7 Mur-Artal R,Montiel J M M,Tardós J D.ORB-SLAM:A versatile and accurate monocular SLAM system.IEEE Transactions on Robotics,2015;31(5):1147-1163
8 Khoshelham K,Elberink S O.Accuracy and resolution of Kinect depth data for indoor mapping applications.Sensors,2012;12(2):1437-1457
9 Dai A,Izadi S,Theobalt C.Bundle fusion:Real-time globally consistent 3D reconstruction using on-the-fly surface re-integration.ACMTransactions on Graphics,2017;36(4):1-17
10陈之杰.基于RGB-D感知的移动机器人三维地图构建与导航研究及应用.南京:东南大学,2016Chen Zhijie.Research and application of mobile robot 3d mapping and navigation with RGB-D perception.Nanjing:Southeast University,2016