基于双目视觉的运动目标跟踪与三维测量
|
摘要
应用双目视觉技术对运动目标进行跟踪,并对其深度和运动信息进行测量是当前计算机视觉研究的热点问题,在军用和民用领域内有着广泛而实际的应用。本文基于双目视觉,以获得目标质心的三维坐标、深度和运动速度为主要目的,研究了运动目标检测与跟踪、双目立体视觉标定、组建双目立体视觉运动目标跟踪与测量实验系统三方面的内容。
在运动目标检测与跟踪方面:重点研究了融合运动目标位置预测的MeanShift算法,并根据Bhattacharyya系数值对目标被遮挡的情况进行了判断;使用本文算法针对目标非匀速运动和发生遮挡的情况进行了计算机仿真实验,在算法运行效率和跟踪稳定性方面获得了预期的效果。
在双目立体视觉标定方面:制作了圆靶标定板并提取了靶面特征点的图像坐标,在分析“TSAI”标定算法的基础上分别完成了左右摄像机的标定;阐述了双目视觉下的空间点三维坐标的提取算法;运用VC++6.0开发了双目立体视觉标定系统,通过测量空间点深度信息和空间物体尺寸大小,验证了系统结果。
在组建双目立体视觉运动目标跟踪与测量实验系统方面:采用代价函数对左右视场下的运动目标进行立体匹配;分析了运动目标深度和速度的测量方法;运用VC++6.0将双目立体视觉标定、运动目标检测与跟踪和目标质心提取三部分内容进行系统的结合,设计了双目立体视觉运动目标跟踪与测量实验系统,使用该系统实现了提取运动目标质心的三维坐标及其深度和速度的测量。
The application of binocular vision techniques for tracking the moving object, and measuring its depth information and movement information is the hot research in computer vision, which used widely in civilian and military. Based on binocular vision, for the purpose to obtain the 3D coordinate, depth information and velocity of the object, this paper mainly discussed the following questions: moving object detecting and tracking, binocular stereo vision calibration, the construction of binocular stereo vision moving object tracking and measuring experimental system :
On moving object detecting and tracking: it mainly researched the combination algorithm of the position-forecasting of moving object in the Mean Shift, using Bhattacharyya coefficients to forecast the occlusion. Then, the simulation is performed when the object is in Non-uniform linear motion and when occlusion happens, which achieved predicted results in running efficiency and tracking stability.
On binocular stereo vision calibration: First, the calibrating board with circles target is made and the coordinate in image of feature Points in it are extracted. Left and right cameras are calibrated by using "TSAI" calibration algorithm. Second, the extraction algorithm of 3D coordinate of spatial points under binocular vision is discussed. Then, a binocular stereo vision calibration system is built by using VC++6.0 and the calibration experiment is carried on. Experimental results are validated by measuring the depth information and the size of spatial object.
On construction of binocular stereo vision moving object tracking and measuring experimental system: First, the moving object under bi-side visions are matched by using cost function and the measurement of depth information and velocity are analyzed. Second, using VC++6.0, combined the parts discussed above and an experimental system for binocular stereo vision moving object tracking and measuring is designed. Then, the centroid of the moving object is extracted, and the depth information and velocity are measured by using the system.
在运动目标检测与跟踪方面:重点研究了融合运动目标位置预测的MeanShift算法,并根据Bhattacharyya系数值对目标被遮挡的情况进行了判断;使用本文算法针对目标非匀速运动和发生遮挡的情况进行了计算机仿真实验,在算法运行效率和跟踪稳定性方面获得了预期的效果。
在双目立体视觉标定方面:制作了圆靶标定板并提取了靶面特征点的图像坐标,在分析“TSAI”标定算法的基础上分别完成了左右摄像机的标定;阐述了双目视觉下的空间点三维坐标的提取算法;运用VC++6.0开发了双目立体视觉标定系统,通过测量空间点深度信息和空间物体尺寸大小,验证了系统结果。
在组建双目立体视觉运动目标跟踪与测量实验系统方面:采用代价函数对左右视场下的运动目标进行立体匹配;分析了运动目标深度和速度的测量方法;运用VC++6.0将双目立体视觉标定、运动目标检测与跟踪和目标质心提取三部分内容进行系统的结合,设计了双目立体视觉运动目标跟踪与测量实验系统,使用该系统实现了提取运动目标质心的三维坐标及其深度和速度的测量。
The application of binocular vision techniques for tracking the moving object, and measuring its depth information and movement information is the hot research in computer vision, which used widely in civilian and military. Based on binocular vision, for the purpose to obtain the 3D coordinate, depth information and velocity of the object, this paper mainly discussed the following questions: moving object detecting and tracking, binocular stereo vision calibration, the construction of binocular stereo vision moving object tracking and measuring experimental system :
On moving object detecting and tracking: it mainly researched the combination algorithm of the position-forecasting of moving object in the Mean Shift, using Bhattacharyya coefficients to forecast the occlusion. Then, the simulation is performed when the object is in Non-uniform linear motion and when occlusion happens, which achieved predicted results in running efficiency and tracking stability.
On binocular stereo vision calibration: First, the calibrating board with circles target is made and the coordinate in image of feature Points in it are extracted. Left and right cameras are calibrated by using "TSAI" calibration algorithm. Second, the extraction algorithm of 3D coordinate of spatial points under binocular vision is discussed. Then, a binocular stereo vision calibration system is built by using VC++6.0 and the calibration experiment is carried on. Experimental results are validated by measuring the depth information and the size of spatial object.
On construction of binocular stereo vision moving object tracking and measuring experimental system: First, the moving object under bi-side visions are matched by using cost function and the measurement of depth information and velocity are analyzed. Second, using VC++6.0, combined the parts discussed above and an experimental system for binocular stereo vision moving object tracking and measuring is designed. Then, the centroid of the moving object is extracted, and the depth information and velocity are measured by using the system.
引文
[1]马颂德,张正友.计算机视觉--计算理论与算法基础[M].北京:科学出版社,1998.
[2]D.Vision,W.H.Freeman and Company San Francisco.姚国正,刘磊,汪云九泽.视觉计算理论.北京:科学出版社,1988.
[3]邵文坤,黄爱民,韦庆.目标跟踪方法综述[J].影像技术.2006,18(1):17-20
[4]Collins Robert T,Lipton Alan J,Takeo Kanade.Introduction to the Special Section on Video Surveillance[J].IEEE Trans.on Pattern Analysis and Machine Intelligence,2000,22(8):745-746.
[5]WEI G Y,LI C F.Design and implementation of visual servoing system for realistic air target tracking[A].Proceedings of IEEE International Conference on Robotics and Automation.2001,vol.1:229-234
[6]张春森.序列立体图像三维运动物体定位与跟踪.武汉大学,博士学位论文,武汉,2004.
[7]刘冬冬.基于双目视觉和CamShift算法的目标检测与跟踪.山东大学,硕士学位论文,济南,2007.
[8]Banard S,Fischler M.Computational Stereo.Computing Surveys,December,1982.14(4):553-570.
[9]Kei Okada,Masayuki Inaba.Integration Of Real-Time Binocular Stereo Vision And Whole Body Information For Dynamic Walking Navigation Of Humanoid Robot.IEEE Conference on Multisensor Fusion and Intergration for Intelligent Systems.2003.
[10]Clark F O,Matthies L.Maximum likelihood rover localization by matching range maps[C].IEEE International Conference on Robotics and Automation,Leuven,Belgium.1998:272-277.
[11]Minoru Asada,Takamaro Tanaka.Visual Tracking of Unknown Moving object By Adaptive Binocular Visual Servoing[C].Proceeding of the 1999 IEEE International Conference on Multisensor Fusion and Integration for Intellugent System,1999
[12]Yajun Fang,Ichiro Masaki,Berthold Horn.Depth-Based Target Segmentation for Intelligent Vehicles:Fusion of Radar and Binocular Stereo.IEEE Transactions On Intelligent Transportation Systems,2002;3(3)
[13]管业鹏,童林夙,陈娜.基于双目立体视觉的偏转线圈测量方法研究.电子学报,2003年09期
[14]郑南宁.计算机视觉与模式识别.北京:国防工业出版社
[15]http://www.aiar.xjtu.edu.cn/pages/research/researching%20fields/index.html
[16]http://sit.nju.edu.cn/works.htm
[17]http://www.visionlab.sjtu.edu.cn/Default.htm
[18]C.Anderson,Peter Burt,G.van der Wal.Change detection and tracking using pyramid transformation techniques.In Proceedings of SPIE-Intelligent Robots and Computer Vision,volume 579,pages 72-78,1985.
[19]I.Haritaoglu,Larry S.Davis,D.Harwood.w4:who? when? where? what? a real time system for detecting and tracking people[C].Japan:International conference on face and gesture recognition,1998
[20]C Wren,A Azarbayejani.T Darrell,et al.Pfinder:Real-time tracking of the human body[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1997,19(7):780-785
[21]J.Barron,D.Fleet,S.Beauchemin.Performance of optical flow techniques[C].International Journal of Computer Vision,1994,12(1),42-77
[22]J.B.Kim,H.J.Kim.Efficient region-based motion segmentation for video monitoring system.[J].Pattern Recognition Letters,2003,(24):113-128
[23]王宾,潘建寿,梁雁冰.基于Matrox卡的视频图像运动检测.西北大学学报(自然科学网络版),2004,3(2).
[24]朱虹.数字图像处理基础1版次.北京:科学出版社.2005.
[25]Y.Bar-Shalon,T.Fortmann.Tracking and Data Association.New Yourk:Academic Press.1988:106.
[26]Fukunaga K,Hostetler L D.The estimation of-the gradient of a density function, with applications in pattern recognition[J].IEEE Transactions on Information Theory,1975,21(1):32-40.
[27]Cheng Yi-zong.Mean Shift,mode seeking,and clustering.IEEE Transactions on Pattern Analysis and Machine lntelligence,1998,17(8):790-799.
[28]王宾,潘建寿,王琳,崔宇巍.基于多特征融合的均值偏移视频目标跟踪算法.小型微型计算机系统,2006年09期.
[29]常发亮等.基于均值漂移与卡尔曼滤波的目标跟踪算法.计算机工程与应用,2007,43(12).
[30]王江涛等.遮挡情况下基于Kalman均值偏移的目标跟踪.系统仿真学报,2007vol.19No.18.
[31]Comaniciu,D.,P Meer.Mean Shift analysis and application,in Proceedings of the Seventh IEEE International Conference.Computer Vision.1999.
[32]Comaniciu,D.,V Ramesh,P.Meer.Real-time tracking of non-rigid objects using Mean Shift.In Computer Vision and Pattern Recognition,2000.Proceedings.IEEE Conference 2000.
[33]Comaniciu,D.,V Ramesh,P.Meer.Kernel-based object tracking.IEEE Transactions.on Pattern Analysis and Machine Intelligence,2003.25(2):p.564-577.
[34]F.Aherne,N.Thacker,P.Rockett.The Bhattacharyya Metric as an Absolute Similarity Measure for Frequency Coded Data.Kybernetika,vol.34,No.4,1998,pp.363-368.
[35]R.Y Tsai.A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses.IEEE Trans.Rob.Autom.3,323-344(1987).
[36]邱茂林,马颂德,李毅.计算机视觉中摄像机定标综述.自动化学报,2000,26(1):43-55.
[37]R.Y Tsai.Accuracy analysis and prediction for 3d robotics vision metrology.Technical Report RC 11348,IBM Re-search,in preparation.
[38]R.Y.TSAI.Review of the two-stage camera calibration techniques plus some new implemention tips and new techniques for center and scale calibration.2nd Topical Meeting on Machine Vision,Optical Society of America,March 18-20,1987.Also IBM RC 12301
[39]ZHANG Z.A Flexible new.technique for camera calibration[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2000,22(11):1330-1334.
[40]Harris.C,Stephens.M.A combined corner and edge detector.Proceedings of the 4th A lvey Vision Conference,1988:147-151.
[41]SMITH S M,BRADY J M.SUSAN.a new approach to low level image processing[J].International Journal of Computer Vision,1997,23(1):45-78.
[42]赵文彬,张艳宁.角点检测技术综述.计算机应用研究,2006年10期.
[43]郭海霞,解凯.角点检测技术的研究.哈尔滨师范大学自然科学学报,2007年02期.
[44]章毓晋.图像分割.北京:科学出版社,2001.
[45]张旭.基于光学图像的航天器目标测距方法研究.哈尔滨工业大学,硕士学位论文,哈尔滨,2006.
[46]陈小天,摄像机标定技术研究,国防科技大学,硕士学位论文,长沙,2003.
[47]于泓,摄像机标定算法研究,山东大学,硕士学位论文,济南,2006.
[48]张伟,程鸿,韦穗.摄像机标定系统的设计与实现.计算机工程,2007年02期
[49]全厚德,闫守成,张洪才.用于交会测量的摄像机标定系统设计.测控技术,2006年03期.
[50]DAVID G.LOWE.Distinctive Image Features from Scale-Invariant Keypoints.International Journal of Computer Vision 60(2),91-110,2004
[51]D.Scharstein,R.Szeliski.A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms.International Journal of Computer Vision 47(1):7-42,2002.
[52]宰小涛.基于SIFT特征描述子的立体匹配算法研究.上海交通大学,硕士学位论文,上海,2007.
[2]D.Vision,W.H.Freeman and Company San Francisco.姚国正,刘磊,汪云九泽.视觉计算理论.北京:科学出版社,1988.
[3]邵文坤,黄爱民,韦庆.目标跟踪方法综述[J].影像技术.2006,18(1):17-20
[4]Collins Robert T,Lipton Alan J,Takeo Kanade.Introduction to the Special Section on Video Surveillance[J].IEEE Trans.on Pattern Analysis and Machine Intelligence,2000,22(8):745-746.
[5]WEI G Y,LI C F.Design and implementation of visual servoing system for realistic air target tracking[A].Proceedings of IEEE International Conference on Robotics and Automation.2001,vol.1:229-234
[6]张春森.序列立体图像三维运动物体定位与跟踪.武汉大学,博士学位论文,武汉,2004.
[7]刘冬冬.基于双目视觉和CamShift算法的目标检测与跟踪.山东大学,硕士学位论文,济南,2007.
[8]Banard S,Fischler M.Computational Stereo.Computing Surveys,December,1982.14(4):553-570.
[9]Kei Okada,Masayuki Inaba.Integration Of Real-Time Binocular Stereo Vision And Whole Body Information For Dynamic Walking Navigation Of Humanoid Robot.IEEE Conference on Multisensor Fusion and Intergration for Intelligent Systems.2003.
[10]Clark F O,Matthies L.Maximum likelihood rover localization by matching range maps[C].IEEE International Conference on Robotics and Automation,Leuven,Belgium.1998:272-277.
[11]Minoru Asada,Takamaro Tanaka.Visual Tracking of Unknown Moving object By Adaptive Binocular Visual Servoing[C].Proceeding of the 1999 IEEE International Conference on Multisensor Fusion and Integration for Intellugent System,1999
[12]Yajun Fang,Ichiro Masaki,Berthold Horn.Depth-Based Target Segmentation for Intelligent Vehicles:Fusion of Radar and Binocular Stereo.IEEE Transactions On Intelligent Transportation Systems,2002;3(3)
[13]管业鹏,童林夙,陈娜.基于双目立体视觉的偏转线圈测量方法研究.电子学报,2003年09期
[14]郑南宁.计算机视觉与模式识别.北京:国防工业出版社
[15]http://www.aiar.xjtu.edu.cn/pages/research/researching%20fields/index.html
[16]http://sit.nju.edu.cn/works.htm
[17]http://www.visionlab.sjtu.edu.cn/Default.htm
[18]C.Anderson,Peter Burt,G.van der Wal.Change detection and tracking using pyramid transformation techniques.In Proceedings of SPIE-Intelligent Robots and Computer Vision,volume 579,pages 72-78,1985.
[19]I.Haritaoglu,Larry S.Davis,D.Harwood.w4:who? when? where? what? a real time system for detecting and tracking people[C].Japan:International conference on face and gesture recognition,1998
[20]C Wren,A Azarbayejani.T Darrell,et al.Pfinder:Real-time tracking of the human body[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1997,19(7):780-785
[21]J.Barron,D.Fleet,S.Beauchemin.Performance of optical flow techniques[C].International Journal of Computer Vision,1994,12(1),42-77
[22]J.B.Kim,H.J.Kim.Efficient region-based motion segmentation for video monitoring system.[J].Pattern Recognition Letters,2003,(24):113-128
[23]王宾,潘建寿,梁雁冰.基于Matrox卡的视频图像运动检测.西北大学学报(自然科学网络版),2004,3(2).
[24]朱虹.数字图像处理基础1版次.北京:科学出版社.2005.
[25]Y.Bar-Shalon,T.Fortmann.Tracking and Data Association.New Yourk:Academic Press.1988:106.
[26]Fukunaga K,Hostetler L D.The estimation of-the gradient of a density function, with applications in pattern recognition[J].IEEE Transactions on Information Theory,1975,21(1):32-40.
[27]Cheng Yi-zong.Mean Shift,mode seeking,and clustering.IEEE Transactions on Pattern Analysis and Machine lntelligence,1998,17(8):790-799.
[28]王宾,潘建寿,王琳,崔宇巍.基于多特征融合的均值偏移视频目标跟踪算法.小型微型计算机系统,2006年09期.
[29]常发亮等.基于均值漂移与卡尔曼滤波的目标跟踪算法.计算机工程与应用,2007,43(12).
[30]王江涛等.遮挡情况下基于Kalman均值偏移的目标跟踪.系统仿真学报,2007vol.19No.18.
[31]Comaniciu,D.,P Meer.Mean Shift analysis and application,in Proceedings of the Seventh IEEE International Conference.Computer Vision.1999.
[32]Comaniciu,D.,V Ramesh,P.Meer.Real-time tracking of non-rigid objects using Mean Shift.In Computer Vision and Pattern Recognition,2000.Proceedings.IEEE Conference 2000.
[33]Comaniciu,D.,V Ramesh,P.Meer.Kernel-based object tracking.IEEE Transactions.on Pattern Analysis and Machine Intelligence,2003.25(2):p.564-577.
[34]F.Aherne,N.Thacker,P.Rockett.The Bhattacharyya Metric as an Absolute Similarity Measure for Frequency Coded Data.Kybernetika,vol.34,No.4,1998,pp.363-368.
[35]R.Y Tsai.A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses.IEEE Trans.Rob.Autom.3,323-344(1987).
[36]邱茂林,马颂德,李毅.计算机视觉中摄像机定标综述.自动化学报,2000,26(1):43-55.
[37]R.Y Tsai.Accuracy analysis and prediction for 3d robotics vision metrology.Technical Report RC 11348,IBM Re-search,in preparation.
[38]R.Y.TSAI.Review of the two-stage camera calibration techniques plus some new implemention tips and new techniques for center and scale calibration.2nd Topical Meeting on Machine Vision,Optical Society of America,March 18-20,1987.Also IBM RC 12301
[39]ZHANG Z.A Flexible new.technique for camera calibration[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2000,22(11):1330-1334.
[40]Harris.C,Stephens.M.A combined corner and edge detector.Proceedings of the 4th A lvey Vision Conference,1988:147-151.
[41]SMITH S M,BRADY J M.SUSAN.a new approach to low level image processing[J].International Journal of Computer Vision,1997,23(1):45-78.
[42]赵文彬,张艳宁.角点检测技术综述.计算机应用研究,2006年10期.
[43]郭海霞,解凯.角点检测技术的研究.哈尔滨师范大学自然科学学报,2007年02期.
[44]章毓晋.图像分割.北京:科学出版社,2001.
[45]张旭.基于光学图像的航天器目标测距方法研究.哈尔滨工业大学,硕士学位论文,哈尔滨,2006.
[46]陈小天,摄像机标定技术研究,国防科技大学,硕士学位论文,长沙,2003.
[47]于泓,摄像机标定算法研究,山东大学,硕士学位论文,济南,2006.
[48]张伟,程鸿,韦穗.摄像机标定系统的设计与实现.计算机工程,2007年02期
[49]全厚德,闫守成,张洪才.用于交会测量的摄像机标定系统设计.测控技术,2006年03期.
[50]DAVID G.LOWE.Distinctive Image Features from Scale-Invariant Keypoints.International Journal of Computer Vision 60(2),91-110,2004
[51]D.Scharstein,R.Szeliski.A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms.International Journal of Computer Vision 47(1):7-42,2002.
[52]宰小涛.基于SIFT特征描述子的立体匹配算法研究.上海交通大学,硕士学位论文,上海,2007.