多传感器时空一致及其信息融合技术研究
|
摘要
论文以陆地自主车(Autonomous Land Vehicle, ALV)为背景,利用研究室已有的自主越野车辆平台,对多传感器信息融合技术中时空一致问题进行了深入的研究,在此基础上实现了激光雷达和立体视觉信息的融合障碍检测。
多传感器时空一致是将各个不同来源的数据变换到同一个时空基准的过程,包括时间同步和空间对准两方面内容。时间同步是把各个传感器的时间统一到参考标准时间上；空间对准则要建立从各个传感器不同坐标系到一个统一的基准坐标系的变换关系(ALV传感器的空间对准必须通过各个传感器间的空间标定来完成)。
本文的主要研究工作包括：
(1)在基于局域网的时间同步方法的基础上,提出了一种用于多传感器系统时间同步的改进算法,该算法通过多台客户端与服务器的数据交换来实现网络中各台计算机的时间同步。仿真实验表明该算法可以在网络延时变化的情况下达到毫秒级以下的时间同步精度。
(2)提出了两种传感器空间标定算法。利用单线激光雷达几何扫描模型,设计了一种激光雷达外标定算法；提出并设计了一种平面模板标定算法,完成了激光雷达和图像的直接标定。实车实验验证了两种算法的有效性。
(3)研究了激光雷达和立体视觉摄像机的信息融合问题,实现了一种基于D-S证据理论的融合障碍检测算法。实车平台的静止实验和运动实验表明该融合算法可以正确的融合激光雷达和立体视觉的信息,获得可靠的环境描述。
Based on Autonomous Land Vehicle (ALV), this thesis investigates the problem of multi-sensor's time-space consistency. Fusion of lidar and stereo vision information is applied to detect obstacles.
Multi-sensor's time-space consistency is to transform different sources of information to the same time-space framework, which includes two aspects:one is the time synchronism and the other is the space alignment. Time synchronism is to unify each sensor's time in the reference standard time; space alignment establishes the relationship of the transformation from different sensor coordinate systems to the reference coordinate system (ALV sensor's space alignment is based on the accurate sensor calibration).
The main works of the thesis are as follows:
(1) An improved time synchronism algorithm is proposed based on local area network in multi-sensor system, the algorithm can synchronize each computer's time through the data exchange between server and client. Simulation results show that our algorithm can meet the time synchronization accuracy need (<1ms) when network delay varys.
(2) Two sensor calibration algorithms are proposed. One is single-line lidar exterior calibration algorithm based on the lidar geometrical model; the other is the lidar and image direct calibration using the board designed by the author. Real vehicle experiments verify the validity of both algorithms.
(3) The fusion problem of the Lidar and stereo vision information is studied using the D-S evidence theory to detect obstacles. Both static and kinetic real vehicle experiments indicate that the algorithm can fuse Lidar and stereo vision information successfully and obtain reliable environment description.
多传感器时空一致是将各个不同来源的数据变换到同一个时空基准的过程,包括时间同步和空间对准两方面内容。时间同步是把各个传感器的时间统一到参考标准时间上；空间对准则要建立从各个传感器不同坐标系到一个统一的基准坐标系的变换关系(ALV传感器的空间对准必须通过各个传感器间的空间标定来完成)。
本文的主要研究工作包括：
(1)在基于局域网的时间同步方法的基础上,提出了一种用于多传感器系统时间同步的改进算法,该算法通过多台客户端与服务器的数据交换来实现网络中各台计算机的时间同步。仿真实验表明该算法可以在网络延时变化的情况下达到毫秒级以下的时间同步精度。
(2)提出了两种传感器空间标定算法。利用单线激光雷达几何扫描模型,设计了一种激光雷达外标定算法；提出并设计了一种平面模板标定算法,完成了激光雷达和图像的直接标定。实车实验验证了两种算法的有效性。
(3)研究了激光雷达和立体视觉摄像机的信息融合问题,实现了一种基于D-S证据理论的融合障碍检测算法。实车平台的静止实验和运动实验表明该融合算法可以正确的融合激光雷达和立体视觉的信息,获得可靠的环境描述。
Based on Autonomous Land Vehicle (ALV), this thesis investigates the problem of multi-sensor's time-space consistency. Fusion of lidar and stereo vision information is applied to detect obstacles.
Multi-sensor's time-space consistency is to transform different sources of information to the same time-space framework, which includes two aspects:one is the time synchronism and the other is the space alignment. Time synchronism is to unify each sensor's time in the reference standard time; space alignment establishes the relationship of the transformation from different sensor coordinate systems to the reference coordinate system (ALV sensor's space alignment is based on the accurate sensor calibration).
The main works of the thesis are as follows:
(1) An improved time synchronism algorithm is proposed based on local area network in multi-sensor system, the algorithm can synchronize each computer's time through the data exchange between server and client. Simulation results show that our algorithm can meet the time synchronization accuracy need (<1ms) when network delay varys.
(2) Two sensor calibration algorithms are proposed. One is single-line lidar exterior calibration algorithm based on the lidar geometrical model; the other is the lidar and image direct calibration using the board designed by the author. Real vehicle experiments verify the validity of both algorithms.
(3) The fusion problem of the Lidar and stereo vision information is studied using the D-S evidence theory to detect obstacles. Both static and kinetic real vehicle experiments indicate that the algorithm can fuse Lidar and stereo vision information successfully and obtain reliable environment description.
引文
[1]Luo R. C., Kay M. G.. Multisensor integration and fusion in intelligent systems. IEEE Transactions on Systems, Man and Cybernetics.1989.19(5):901-931.
[2]Shafer S. A., Stentz A., Thorpe C. E.. An Architecture for Sensor Fusion in a Mobiie Robot. Carnegie-Mellon University, Pittsburgh, Report, CMU-RI-TR-86-9, 1986.
[3]Bertozzi M., Broggi A.. GOLD:a parallel real-time stereo vision system for generic obstacle and lane detection. IEEE Transactions on Image Processing.1998.7(1):62-81.
[4]Hellstmm T.. Autonmous navigation for forest machines. Umea University, Umea, Report, UMINF-02-13,2002.
[5]Sebastian T., Mike M., Hendrik D., David S., Andrei A., James D., Philip F., John G., Morgan H., Gabriel H., Kenny L., Celia O., Mark P., Vaughan P., Pascal S., Sven S., Cedric D., Lars-Erik J., Christian K., Charles M., Carlo R., Joe van N., Eric J., Philippe A., Gary B., Bob D., Scott E., Adrian K., Ara N., Pamela M.. Stanley:The robot that won the DARPA Grand Challenge. Journal of Field Robotics.2006.23(9):661-692.
[6]Urmson C., Anhalt J., Clark M., Koon P.. A Robust Approach to High-Speed Navigation for Unrehearsed Desert Terrain. Journal of Field Robotics.2006.23(8): 467-508.
[7]Ferguson D., Darms M., Urmson C., Kolski S.. Detection, prediction, and avoidance of dynamic obstacles in urban environments. IEEE Intelligent Vehicles Symposium, Eindhoven,The Netherlands,2008:1149-1154.
[8]贺席兵.信息融合中多平台多传感器的时空对准研究,硕士学位论文.西安：西北工业大学,2001.
[9]刘同明,夏祖勋,解洪成.数据融合技术及其应用.北京：国防工业出版社,1998.
[10]韩崇昭,朱洪艳,段战胜.多源信息融合.北京：清华大学出版社,2006.
[11]Hackett J. K., Shah M.. Multi-sensor fusion:a perspective. Proceedings of IEEE International Conference on Robotics and Automation,1990:1324-1330
[12]李玉榕.信融合与智能处理研究,博士学位论文.杭州：浙江大学,2001.
[13]Luo R. C., Lin M. H., Scherp R. S.. Dynamic multi-sensor data fusion system for intelligent robots. IEEE Journal of Robotics and Automation.1988.4(4):386-396.
[14]Dempster A. P.. Upper and Lower Probabilities Induced by a Multi-valued Mapping. The Annals of Mathematical Statistics.1967.328:325-339.
[15]Shafer G.. A Mathematical Theory of Evidence. Princeton:Princeton University Press,1976.
[16]张奇,顾伟康.基于多传感器数据融合的环境理解及障碍物检测算法.机器人.1998.20(2)：104-110.
[17]张奇,顾伟康,刘济林.基于Dempster-Shafer证据推理理论的ALV视觉信息融合.计算机学报.1999.22(2)：193-198.
[18]Zhiyu X.. An environmental perception system to autonomous off-road navigation by using multi-sensor data fusion. International Conference on Neural Networks and Brain,2005:1221-1226.
[19]Zhiyu X., Zezhong X., Jilin L.. Small obstacle detection for autonomous land vehicle under semi-structural environments. Proceedings of IEEE Intelligent Transportation Systems,2003:293-298.
[20]Niwa S., Masuda T., Sezaki Y.. Kalman filter with time-variable gain for a multisensor fusion system. Proceedings of IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, Taiwan,1999:56-61.
[21]司现军,王志良.移动机器人多传感器信息融合技术综述.机电工程.2004.21(2)：1-5.
[22]朱晓芸,杨建刚,何志钧.神经网络的多传感器数据融合基于新算法在障碍物识别中的应用.机器人.1997.19(3)：166-172.
[23]Lazar A., Sethi I. K.. Decision Rule Extraction from Trained Neural Networks Using Rough Sets. Intelligent Engineering Systems Through Artificial Neural Networks.1999.9:493-498.
[24]Douglas E. C.,林瑶,蒋慧,杜蔚轩等译.用TCP/IP进行网际互联—第一卷：原理、协议与结构(第四版).北京：电子工业出版社,2001.
[25]汪文俊,龚育昌,朱建明.基于UDP的局域网内时钟同步协议.计算机应用与软件.2007.24(5)：133-135.
[26]Mills D. L.. Internet time synchronization:the network time protocol. IEEE Transactions on Communications.1991.39(10):1482-1493.
[27]杨晓谙,周春晖.网络时间同步分析.计算机时代.2006.(5)：9-11.
[28]赵英,刘冬梅,郭树印,王雪晶.基于NTP的网络时间服务测量.计算机工程与应用.2006.：99-100,119.
[29]田海燕,朱利,王换招,马臻.BOND:一种双向的单路网络延时测量方法.小型微型计算机系统.2005.26(3)：396-399.
[30]郑芳.基于视觉的激光传感器标定算法研究,硕士学位论文.长沙：国防科学技术大学,2005.
[31]Qilong Z., Pless R.. Extrinsic calibration of a camera and laser range finder (improves camera calibration). Proceedings of Proceedings ofConference on Intelligent Robots and Systems,2004:2301-2306.
[32]Unnikrishnan R., Hebert M.. Fast Extrinsic Calibration of a Laser Rangefinder to a Camera. Carnegie Mellon University, Pittsburgh, Report, CMU-RI-TR-05-09, 2005.
[33]Qilong Z., Pless R.. Constraints for Heterogeneous Sensor Auto-Calibration. Proceedings of the Conference on Computer Vision and Pattern Recognition Workshop,2004:38-38.
[34]Pless R., Zhang Q.. Extrinsic Auto-calibration of a Camera and Laser Range Finder. Washington University, St Louis, Report, WUSCE-2003-59,2003.
[35]肖业伦.航空航天器运动的建模——飞行动力学的理论基础.北京：北京航空航天大学出版社,2003.
[36]胡小平,吴美平,王海丽.导弹飞行力学基础.长沙：国防科技大学出版社,2006.
[37]张奇,刘济林,郭小军,顾伟康.激光测距成像雷达三维测距：存在问题及解决方法.浙江大学学报(自然科学版).1998.32(6)：732-738.
[38]庄严.移动机器人基于多传感器数据融合的定位及地图创建研究,博士学位论文.大连：大连理工大学,2004.
[39]Cang Y., Borenstein J.. Characterization of a 2D laser scanner for mobile robot obstacle negotiation. Proceedings of IEEE International Conference on Robotics and Automation, Washington DC,2002:2512-2518.
[40]于金霞,蔡自兴.移动机器人导航中激光雷达测距性能研究.传感技术学报.2006.19(2)：356-360.
[41]Kay S.M.,罗鹏飞等译.统计信号处理基础——估计与检测理论.北京：电子工业出版社,2003.
[42]David A. F., Ponce J.,林学(?),王宏等译.计算机视觉——一种现代的方法.北京：电子工业出版社,2004.
[43]刘欣.ALV立体视觉导航技术研究,硕士学位论文.长沙：国防科学技术大学,2005.
[44]王凌.智能优化算法及其应用.北京：清华大学出版社,2001.
[45]吴维一.激光雷达及多传感器融合技术应用研究,硕士学位论文.长沙：国防科学技术大学,2006.
[46]陈华华.视觉导航关键技术研究：立体视觉和路径规划,博士学位论文.杭州：浙江大学,2005.
[47]何友,王国宏,彭应宁.多传感器信息融合及应用.北京：电子工业出版社,2000.
[48]Zhengyou Z.. Flexible camera calibration by viewing a plane from unknown orientations. The Proceedings of the Seventh IEEE International Conference on Computer Vision,1999:666-673.
[49]黄爱民,安向京,骆力.数字图像处理与分析基础.北京：中国水利水电出版社,2005.
[50]林成森.数值计算方法.北京：科学出版社(第二版),2005.
[51]施妙根,顾丽珍.科学和工程计算基础.北京：清华大学出版社,1999.
[52]项志宇.基于激光雷达移动机器人障碍检测和自定位,博士学位论文.杭州：浙江大学,2002.第64页
[2]Shafer S. A., Stentz A., Thorpe C. E.. An Architecture for Sensor Fusion in a Mobiie Robot. Carnegie-Mellon University, Pittsburgh, Report, CMU-RI-TR-86-9, 1986.
[3]Bertozzi M., Broggi A.. GOLD:a parallel real-time stereo vision system for generic obstacle and lane detection. IEEE Transactions on Image Processing.1998.7(1):62-81.
[4]Hellstmm T.. Autonmous navigation for forest machines. Umea University, Umea, Report, UMINF-02-13,2002.
[5]Sebastian T., Mike M., Hendrik D., David S., Andrei A., James D., Philip F., John G., Morgan H., Gabriel H., Kenny L., Celia O., Mark P., Vaughan P., Pascal S., Sven S., Cedric D., Lars-Erik J., Christian K., Charles M., Carlo R., Joe van N., Eric J., Philippe A., Gary B., Bob D., Scott E., Adrian K., Ara N., Pamela M.. Stanley:The robot that won the DARPA Grand Challenge. Journal of Field Robotics.2006.23(9):661-692.
[6]Urmson C., Anhalt J., Clark M., Koon P.. A Robust Approach to High-Speed Navigation for Unrehearsed Desert Terrain. Journal of Field Robotics.2006.23(8): 467-508.
[7]Ferguson D., Darms M., Urmson C., Kolski S.. Detection, prediction, and avoidance of dynamic obstacles in urban environments. IEEE Intelligent Vehicles Symposium, Eindhoven,The Netherlands,2008:1149-1154.
[8]贺席兵.信息融合中多平台多传感器的时空对准研究,硕士学位论文.西安：西北工业大学,2001.
[9]刘同明,夏祖勋,解洪成.数据融合技术及其应用.北京：国防工业出版社,1998.
[10]韩崇昭,朱洪艳,段战胜.多源信息融合.北京：清华大学出版社,2006.
[11]Hackett J. K., Shah M.. Multi-sensor fusion:a perspective. Proceedings of IEEE International Conference on Robotics and Automation,1990:1324-1330
[12]李玉榕.信融合与智能处理研究,博士学位论文.杭州：浙江大学,2001.
[13]Luo R. C., Lin M. H., Scherp R. S.. Dynamic multi-sensor data fusion system for intelligent robots. IEEE Journal of Robotics and Automation.1988.4(4):386-396.
[14]Dempster A. P.. Upper and Lower Probabilities Induced by a Multi-valued Mapping. The Annals of Mathematical Statistics.1967.328:325-339.
[15]Shafer G.. A Mathematical Theory of Evidence. Princeton:Princeton University Press,1976.
[16]张奇,顾伟康.基于多传感器数据融合的环境理解及障碍物检测算法.机器人.1998.20(2)：104-110.
[17]张奇,顾伟康,刘济林.基于Dempster-Shafer证据推理理论的ALV视觉信息融合.计算机学报.1999.22(2)：193-198.
[18]Zhiyu X.. An environmental perception system to autonomous off-road navigation by using multi-sensor data fusion. International Conference on Neural Networks and Brain,2005:1221-1226.
[19]Zhiyu X., Zezhong X., Jilin L.. Small obstacle detection for autonomous land vehicle under semi-structural environments. Proceedings of IEEE Intelligent Transportation Systems,2003:293-298.
[20]Niwa S., Masuda T., Sezaki Y.. Kalman filter with time-variable gain for a multisensor fusion system. Proceedings of IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, Taiwan,1999:56-61.
[21]司现军,王志良.移动机器人多传感器信息融合技术综述.机电工程.2004.21(2)：1-5.
[22]朱晓芸,杨建刚,何志钧.神经网络的多传感器数据融合基于新算法在障碍物识别中的应用.机器人.1997.19(3)：166-172.
[23]Lazar A., Sethi I. K.. Decision Rule Extraction from Trained Neural Networks Using Rough Sets. Intelligent Engineering Systems Through Artificial Neural Networks.1999.9:493-498.
[24]Douglas E. C.,林瑶,蒋慧,杜蔚轩等译.用TCP/IP进行网际互联—第一卷：原理、协议与结构(第四版).北京：电子工业出版社,2001.
[25]汪文俊,龚育昌,朱建明.基于UDP的局域网内时钟同步协议.计算机应用与软件.2007.24(5)：133-135.
[26]Mills D. L.. Internet time synchronization:the network time protocol. IEEE Transactions on Communications.1991.39(10):1482-1493.
[27]杨晓谙,周春晖.网络时间同步分析.计算机时代.2006.(5)：9-11.
[28]赵英,刘冬梅,郭树印,王雪晶.基于NTP的网络时间服务测量.计算机工程与应用.2006.：99-100,119.
[29]田海燕,朱利,王换招,马臻.BOND:一种双向的单路网络延时测量方法.小型微型计算机系统.2005.26(3)：396-399.
[30]郑芳.基于视觉的激光传感器标定算法研究,硕士学位论文.长沙：国防科学技术大学,2005.
[31]Qilong Z., Pless R.. Extrinsic calibration of a camera and laser range finder (improves camera calibration). Proceedings of Proceedings ofConference on Intelligent Robots and Systems,2004:2301-2306.
[32]Unnikrishnan R., Hebert M.. Fast Extrinsic Calibration of a Laser Rangefinder to a Camera. Carnegie Mellon University, Pittsburgh, Report, CMU-RI-TR-05-09, 2005.
[33]Qilong Z., Pless R.. Constraints for Heterogeneous Sensor Auto-Calibration. Proceedings of the Conference on Computer Vision and Pattern Recognition Workshop,2004:38-38.
[34]Pless R., Zhang Q.. Extrinsic Auto-calibration of a Camera and Laser Range Finder. Washington University, St Louis, Report, WUSCE-2003-59,2003.
[35]肖业伦.航空航天器运动的建模——飞行动力学的理论基础.北京：北京航空航天大学出版社,2003.
[36]胡小平,吴美平,王海丽.导弹飞行力学基础.长沙：国防科技大学出版社,2006.
[37]张奇,刘济林,郭小军,顾伟康.激光测距成像雷达三维测距：存在问题及解决方法.浙江大学学报(自然科学版).1998.32(6)：732-738.
[38]庄严.移动机器人基于多传感器数据融合的定位及地图创建研究,博士学位论文.大连：大连理工大学,2004.
[39]Cang Y., Borenstein J.. Characterization of a 2D laser scanner for mobile robot obstacle negotiation. Proceedings of IEEE International Conference on Robotics and Automation, Washington DC,2002:2512-2518.
[40]于金霞,蔡自兴.移动机器人导航中激光雷达测距性能研究.传感技术学报.2006.19(2)：356-360.
[41]Kay S.M.,罗鹏飞等译.统计信号处理基础——估计与检测理论.北京：电子工业出版社,2003.
[42]David A. F., Ponce J.,林学(?),王宏等译.计算机视觉——一种现代的方法.北京：电子工业出版社,2004.
[43]刘欣.ALV立体视觉导航技术研究,硕士学位论文.长沙：国防科学技术大学,2005.
[44]王凌.智能优化算法及其应用.北京：清华大学出版社,2001.
[45]吴维一.激光雷达及多传感器融合技术应用研究,硕士学位论文.长沙：国防科学技术大学,2006.
[46]陈华华.视觉导航关键技术研究：立体视觉和路径规划,博士学位论文.杭州：浙江大学,2005.
[47]何友,王国宏,彭应宁.多传感器信息融合及应用.北京：电子工业出版社,2000.
[48]Zhengyou Z.. Flexible camera calibration by viewing a plane from unknown orientations. The Proceedings of the Seventh IEEE International Conference on Computer Vision,1999:666-673.
[49]黄爱民,安向京,骆力.数字图像处理与分析基础.北京：中国水利水电出版社,2005.
[50]林成森.数值计算方法.北京：科学出版社(第二版),2005.
[51]施妙根,顾丽珍.科学和工程计算基础.北京：清华大学出版社,1999.
[52]项志宇.基于激光雷达移动机器人障碍检测和自定位,博士学位论文.杭州：浙江大学,2002.第64页