视频中多目标检测和跟踪算法研究
|
摘要
目标跟踪是图像处理、模式识别、计算机视觉等领域的研究热点,在视频监控、智能交通和军事等方面有广泛应用。本文在对摄像机采集的图像序列分析处理基础上,实现视频中人体目标的自动提取、识别和跟踪。在单摄像机条件下,对于视频监控中的多人跟踪问题,提出了一种基于高斯概率模型的算法。基于目标颜色的统计特征,采用改进的K均值方法,将目标区域按颜色信息聚类,并根据聚类结果对目标区域分块,然后用高斯模型对各分块分别进行建模。同时,对目标的位置信息也进行高斯建模。通过计算待检测目标与模型之间颜色和位置的最大联合概率值来实现跟踪。利用前后帧中目标的位置信息及颜色信息,能克服目标遮挡后因信息的丢失而跟踪失败的情况。多摄像机的使用可以扩大监视区域,在多摄像机下的多目标跟踪,需要对同一目标在不同摄像机下的对应,该对应关系可以根据多摄像机的几何信息来确定。本文提出了利用三维信息的方法,以及这种方法的改进,即仅使用极线约束和目标直方图匹配,可以准确而快速的进行不同摄像机中同一目标的对应。在利用多摄像机的几何信息之前,需要对其进行标定,本文提出了一种基于视频序列的RANSAC自动外参求解方法,该方法解决了现存的外参自动估算方法中普遍存在的特征点完全取决于场景而导致算法精度不稳定的缺点,利用多人在场景中走动的方法,控制和提取特征点对,根据可控特征点对的动态性,以及时间上对可控特征点对的积累,采用RANSAC算法有效的剔除外点从而大幅度提升了外参求解的精度。最后,本文利用室外场景条件下拍摄的视频序列对算法进行了测试,并给出了实验结果。
Object tracking has become an important research field in several domains such as image processing, pattern recognition and computer vision. It prevails in diverse applications in visual surveillance, intelligent traffic and military affairs. This dissertation accomplishes automatic detection and pedestrians tracking based on the analysis of video sequences. To deal with the pedestrians tracking problem in video surveillance, an algorithm based on Gaussian probability is proposed. Modeling object on the basis of statistical characteristics of color information which is grouped by improved k-means algorithm, it segments object into blocks according to the grouping results, and utilizes a Gaussian function in order to describe the distribution of color information in each block as well as the position information of pedestrians. The algorithm accomplishes when the maximum joint probability of color and position is obtained. Along with position information and color information of objects among adjacent frames, the failure case which is because of information loss due to occlusion could be solved. The multi-cameras are used to extend the range of surveillance. For multi-object tracking problem, the same pedestrian in different camera should be verified using the geometric information between cameras. The paper proposes an approach which adopts 3D information, and the improvement of the approach which uses epipolar line constraint and histogram matching. The multi-cameras should be calibrated before the geometric information is imported. In order to overcome the disadvantage in current algorithms, a new algorithm called RANSAC external parameters calculation based on videos is developed. While people are walking around, feature point correspondences could be extracted under control. By using the accumulation of correspondences along the sequence, this algorithm removed the outliers and end up with an accurate result. Not only RANSAC but also the dynamic effect of feature points is used to get an accurate result. Finally, the algorithm is tested with videos taken in outdoor environment, and the results are provided in this dissertation.
Object tracking has become an important research field in several domains such as image processing, pattern recognition and computer vision. It prevails in diverse applications in visual surveillance, intelligent traffic and military affairs. This dissertation accomplishes automatic detection and pedestrians tracking based on the analysis of video sequences. To deal with the pedestrians tracking problem in video surveillance, an algorithm based on Gaussian probability is proposed. Modeling object on the basis of statistical characteristics of color information which is grouped by improved k-means algorithm, it segments object into blocks according to the grouping results, and utilizes a Gaussian function in order to describe the distribution of color information in each block as well as the position information of pedestrians. The algorithm accomplishes when the maximum joint probability of color and position is obtained. Along with position information and color information of objects among adjacent frames, the failure case which is because of information loss due to occlusion could be solved. The multi-cameras are used to extend the range of surveillance. For multi-object tracking problem, the same pedestrian in different camera should be verified using the geometric information between cameras. The paper proposes an approach which adopts 3D information, and the improvement of the approach which uses epipolar line constraint and histogram matching. The multi-cameras should be calibrated before the geometric information is imported. In order to overcome the disadvantage in current algorithms, a new algorithm called RANSAC external parameters calculation based on videos is developed. While people are walking around, feature point correspondences could be extracted under control. By using the accumulation of correspondences along the sequence, this algorithm removed the outliers and end up with an accurate result. Not only RANSAC but also the dynamic effect of feature points is used to get an accurate result. Finally, the algorithm is tested with videos taken in outdoor environment, and the results are provided in this dissertation.
引文
[1]王其聪.复杂观测条件下的基于粒子滤波的视觉跟踪[博士学位论文].浙江大学.2007.
[2]Collins R,Lipton A J,Kanade T.A system for video surveillance and monitoring:VSAM final report.Technical Report:CMU-RI-TR-00-12,Carnegie Melon University,Pittsburgh,Peen,America,2000.
[3]Haritaoglu I,Harwood D,Davis L S.W4:Real-time surveillance of people and their activities.IEEE Transactions on Pattern Analysis and Machine Intelligence,2000,22(8):809-830.
[4]Siebel N T,Maybank S J.The ADVISOR visual surveillance system.Proceedings of the ECCV Workshop on Applications of Computer Vision.Prague.May,2004:103-111.
[5]Pai C,Tyan H,Llang Y,Liao H M,Chen S.Pedestrian detection and tracking at crossroads,Pattern Recognition,2004,37(5):1025-1034.
[6]Masoud O,Papankolopoulos N P.A novel method for tracking and counting pedestrians in real-time using a single camera,IEEE Transactions on Vehicular Technology,2001,50(5):1267-1278.
[7]蒋志凯.数字滤波与卡尔曼滤波.北京:中国科学技术出版社.1993.
[8]Isard M,Blake A.Condensation-conditional density propagation for visual tracking.International Journal of Computer Vision.Aug.,1998,29(1):5-28.
[9]Pavlovic V,Rehg J M,Cham T-J,Murphy K P.A dynamic bayesian network approach to figure tracking using learned dynamic models.Proceedings of the Seventh IEEE International Conference on Computer Vision.1999,vol.1:94-101.
[10]Aggarwal J K,Cai Q.Human motion analysis:a review.Computer Vision and Image Understanding,1999,73(3):428-440.
[11]Gavrila D M.The visual analysis of human movement:a survey.Computer Vision and Image Understanding.1999,73(1):82-98.
[12]Moeslund T.B,Granum.E.A survey of computer vision-based human motion capture.Compute Vision.Image Underst.2001,81(3):231-268.
[13]王亮,胡卫明,谭铁牛.人运动的视觉分析综述.计算机学报.2002,25(3):225-237.
[14]Karaulova I,Hall P and Marshall A.A hierarchical model of dynamics for tracking people with a single video camera.In:British Machine Vision Conference,Bristol,UK,2000:352-361.
[15]Ju S,Black M and Yaccob Y.Cardboard people:a parameterized model of articulated image motion.In:Proc IEEE International Conference on Automatic Face and gesture Recognition,Killington,Vermont USA,1996:38-44.
[16]Rohr K.Towards model-based recognition of human movements in image sequences.CV GIP:Image Understanding,1994,59(1):94-115.
[17]Mittal A,Davis L S.M2tracker:A multi-view approach to segmenting and tracking people in a cluttered scene using region-based stereo.International Journal of Computer Vision.2003,51(3):189-203.
[18]Wren C R,Azarbayejani A,Darrell T,Pentland A P.Pfinder:real-time tracking of the human body.IEEE Transactions on Pattern Analysis and Machine Intelligence.1997,19(7):780-785.
[19]McKenna S et al,Tracking groups of people.Computer Vision and Image Understanding,2000,80(1):42-56.
[20]Polana R,Nelson R.Low level recognition of human motion.Proceedings of the IEEE Workshop on Motion of Non-Rigid and Articulated Objects.Austin,1994:77-82.
[21]Jang D-S and Choi H-I.Active models for tracking moving objects.Pattern Recognition,2000,33(7):1135-1146.
[22]刘重庆.基于网格模型的视频对象跟踪研究与应用[博士学位论文].上海交通大学.2003.
[23]Baule L.MPEG4 optimization model.Version 2.0.In:ISO/IECJTC1/SC29/WG11 N3675.October,2000.
[24]刘李杰,蔡德钧,翁南衫.一种面向运动的视频对象分割算法.计算机学报.2000,23(12):1326-1331.
[25]Schwefel H P.Evolution and Optimum Seeking.NY:John Wiley & Sons.1995.
[26]Kass M,Witkinm A.Terzopoulos D.Snakes:active contour models.International Journal on Computer Vision.1988,4(1):321-331.
[27]Kim W,Lee C Y,Lee J J.Tracking moving object using snake's jump based on image Flow.MECHATORNICS.2001(11):199-226.
[28]Chang C,Ansari R.Kernel particle filter for visual tracking.IEEE Signal Processing Letters.2005,12(3):242-245.
[29]Delamarre Q,Faugeras O.3D articulated models and multi-view tracking with silhouettes.Proceedings of the 7th International Conference on Computer Vision.Kerkyra,Greece,1999.
[30]Cai Q,Aggarwal J.Tracking human motion using multiple cameras.Proceedings of the International Conference on Pattern Recognition.Vienna,Austria,1996,vol.3:68-72.
[31]Gennery D.Tracking known three-dimensional objects.Proceedings of the International Conference on American Association of Artificial Intelligence.August,1982:13-17,
[32]Gavrila D,Davis L.3-D model-based tracking of humans in action:a multi-view approach.In:Proc IEEE Conference on Computer Vision and Pattern Recognition,San Francisco,1996:73-80.
[33]Deutscher J,Blake A,Reid I.Articulated body motion capture by annealed particle filtering.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Hilton Head,USA,2000,vol.2:126-133.
[34]Deutscher J,Davison A,Reid I.Automatic partitioning of high dimensional search spaces associated with articulated body motion capture.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Kauai,Hawaii,2001,vol.2:669-676.
[35]陈睿,刘国翊,赵国英,张俊,李华.基于序列蒙特卡罗方法的3D人体运动跟踪.计算机辅助设计与图形学学报.2005,17(1):85-92.
[36]Lee M W,Cohen I,Jung S K.Particle filter with analytical inference for human body tracking.Workshop on Motion and Video Computing.Orlando,Florida,2002:159-165.
[37]胡士强.敬忠良.粒子滤波算法综述[J].控制与决策.2005.20(4):361-365.
[38]Haritaoglu I,Harwood D,Davis L.Multiple People Detection and Tracking Using Silhouettes[C].Fort Collins,Colorado:Proc 2nd IEEE Workshop on Visual Surveillance,1999.6-13.
[39]杜歆,陈建娟,王选贺.一种基于高斯概率模型的多人跟踪算法.传感技术学报.2009.已录用.
[40]Elgammal A E,Davis L S.Probabilistic framework for segmenting people under occlusion[C].IEEE International Conference on Computer Vision,2001,2:145-152.
[41]Prati A,Mikic I,Trivedi M M,Cucchiara R.Detecting moving shadows:formulation,algorithms and evaluation.IEEE Trans.Pattern Anal.Mach.Intell..2003,25(7):918-924.
[42]张莉.多摄像机人体跟踪技术的研究[博士学位论文].浙江大学.2008
[43]Koller D,Daniilidis K,Nagel H.Model-based object tracking in monocular image sequences of road traffic scenes.International Journal of Computer Vision.1993,10(3):257-281.
[44]Cucchiara R,Grana C,Piccardi M,Prati A.Detecting moving objects,ghosts and shadows in video streams.IEEE Transactions on Pattern Analysis and Machine Intelligence.2003,25(10):1337-1342.
[45]Horprasert T,Harwood D,Davis L S.A statistical approach for real-time robust background subtraction and shadow detection.Proceedings of the International Workshop on ICCV FRAME-RATE.Kerkyra,Greece,Sep.,1999:1-19.
[46]Xia S X,Li W C,Zhou Y.Improved k-means clustering algorithm[J].Journal of Southeast University(English Edition),2007,23(3),435-438.
[47]Ess,A.,B.Leibe,et al.(2008).A Mobile Vision System for Robust Multi-Person Tracking.Computer Vision and Pattern Recognition,2008.CVPR 2008.IEEE Conference on:1-8.
[48]Z.Zhang.Flexible Camera Calibration By Viewing a Plane From Unknown Orientations.International Conference on Computer Vision(ICCV'99),Corfu,Greece,Sep.,1999:666-673.
[49]Richard Hartley,Multiple View Geometry in Computer Vision(Second Edition) 2003.
[50]陈曦.陈建娟.谢立.一种基于RANSAC监视系统外参估算方法.哈尔滨工业大学学报.2009.已录用.
[51]Lowe D.Distinctive image features from scale-invariant keypoints.International Journal of Computer Vision.2004,60(2):91-110.
[52]Lindeberg Tony.Feature detection with automatic scale selection[J].International Journal of Computer Vision(S0920-5691),1998,30(2):79-116.
[53]Lowe,D.Distinctive image features from scale-invariant key points.International Journal of Computer Vision,2004,60(2):91-110.
[54]Martin A.Fischler and Robert C.Bolles(June 1981)."Random Sample Consensus:A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography".Comm.of the ACM 24:381-395.doi:10.1145/358669.358692.
[55]Javed O,Rasheed Z,Shafique K,Shah M.Tracking across multiple cameras with disjoint views.Proceedings of the Ninth IEEE International Conference on Computer Vision.2003,vol.2:952-957.
[56]Khan S,Shah M.Consistent labeling of tracked objects in multiple cameras with overlapping fields of view.IEEE Transactions on Pattern Analysis and Machine Intelligence.October 2003,25(10):1355-1360.
[57]Shi J,Tomashi C.Good feature to track.Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.USA,1994(2):593-600.
[58]Brown M,Burschka D,Hager G.Advances in computational stereo.IEEE Transactions on Pattern Analysis and Machine Intelligence.Aug,2003,25(8):993-1008.
[59]朱磊.一种基于直方图统计特征的直方图匹配算法的研究[J].计算技术与自动化,2004,23(2):48—51.
[2]Collins R,Lipton A J,Kanade T.A system for video surveillance and monitoring:VSAM final report.Technical Report:CMU-RI-TR-00-12,Carnegie Melon University,Pittsburgh,Peen,America,2000.
[3]Haritaoglu I,Harwood D,Davis L S.W4:Real-time surveillance of people and their activities.IEEE Transactions on Pattern Analysis and Machine Intelligence,2000,22(8):809-830.
[4]Siebel N T,Maybank S J.The ADVISOR visual surveillance system.Proceedings of the ECCV Workshop on Applications of Computer Vision.Prague.May,2004:103-111.
[5]Pai C,Tyan H,Llang Y,Liao H M,Chen S.Pedestrian detection and tracking at crossroads,Pattern Recognition,2004,37(5):1025-1034.
[6]Masoud O,Papankolopoulos N P.A novel method for tracking and counting pedestrians in real-time using a single camera,IEEE Transactions on Vehicular Technology,2001,50(5):1267-1278.
[7]蒋志凯.数字滤波与卡尔曼滤波.北京:中国科学技术出版社.1993.
[8]Isard M,Blake A.Condensation-conditional density propagation for visual tracking.International Journal of Computer Vision.Aug.,1998,29(1):5-28.
[9]Pavlovic V,Rehg J M,Cham T-J,Murphy K P.A dynamic bayesian network approach to figure tracking using learned dynamic models.Proceedings of the Seventh IEEE International Conference on Computer Vision.1999,vol.1:94-101.
[10]Aggarwal J K,Cai Q.Human motion analysis:a review.Computer Vision and Image Understanding,1999,73(3):428-440.
[11]Gavrila D M.The visual analysis of human movement:a survey.Computer Vision and Image Understanding.1999,73(1):82-98.
[12]Moeslund T.B,Granum.E.A survey of computer vision-based human motion capture.Compute Vision.Image Underst.2001,81(3):231-268.
[13]王亮,胡卫明,谭铁牛.人运动的视觉分析综述.计算机学报.2002,25(3):225-237.
[14]Karaulova I,Hall P and Marshall A.A hierarchical model of dynamics for tracking people with a single video camera.In:British Machine Vision Conference,Bristol,UK,2000:352-361.
[15]Ju S,Black M and Yaccob Y.Cardboard people:a parameterized model of articulated image motion.In:Proc IEEE International Conference on Automatic Face and gesture Recognition,Killington,Vermont USA,1996:38-44.
[16]Rohr K.Towards model-based recognition of human movements in image sequences.CV GIP:Image Understanding,1994,59(1):94-115.
[17]Mittal A,Davis L S.M2tracker:A multi-view approach to segmenting and tracking people in a cluttered scene using region-based stereo.International Journal of Computer Vision.2003,51(3):189-203.
[18]Wren C R,Azarbayejani A,Darrell T,Pentland A P.Pfinder:real-time tracking of the human body.IEEE Transactions on Pattern Analysis and Machine Intelligence.1997,19(7):780-785.
[19]McKenna S et al,Tracking groups of people.Computer Vision and Image Understanding,2000,80(1):42-56.
[20]Polana R,Nelson R.Low level recognition of human motion.Proceedings of the IEEE Workshop on Motion of Non-Rigid and Articulated Objects.Austin,1994:77-82.
[21]Jang D-S and Choi H-I.Active models for tracking moving objects.Pattern Recognition,2000,33(7):1135-1146.
[22]刘重庆.基于网格模型的视频对象跟踪研究与应用[博士学位论文].上海交通大学.2003.
[23]Baule L.MPEG4 optimization model.Version 2.0.In:ISO/IECJTC1/SC29/WG11 N3675.October,2000.
[24]刘李杰,蔡德钧,翁南衫.一种面向运动的视频对象分割算法.计算机学报.2000,23(12):1326-1331.
[25]Schwefel H P.Evolution and Optimum Seeking.NY:John Wiley & Sons.1995.
[26]Kass M,Witkinm A.Terzopoulos D.Snakes:active contour models.International Journal on Computer Vision.1988,4(1):321-331.
[27]Kim W,Lee C Y,Lee J J.Tracking moving object using snake's jump based on image Flow.MECHATORNICS.2001(11):199-226.
[28]Chang C,Ansari R.Kernel particle filter for visual tracking.IEEE Signal Processing Letters.2005,12(3):242-245.
[29]Delamarre Q,Faugeras O.3D articulated models and multi-view tracking with silhouettes.Proceedings of the 7th International Conference on Computer Vision.Kerkyra,Greece,1999.
[30]Cai Q,Aggarwal J.Tracking human motion using multiple cameras.Proceedings of the International Conference on Pattern Recognition.Vienna,Austria,1996,vol.3:68-72.
[31]Gennery D.Tracking known three-dimensional objects.Proceedings of the International Conference on American Association of Artificial Intelligence.August,1982:13-17,
[32]Gavrila D,Davis L.3-D model-based tracking of humans in action:a multi-view approach.In:Proc IEEE Conference on Computer Vision and Pattern Recognition,San Francisco,1996:73-80.
[33]Deutscher J,Blake A,Reid I.Articulated body motion capture by annealed particle filtering.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Hilton Head,USA,2000,vol.2:126-133.
[34]Deutscher J,Davison A,Reid I.Automatic partitioning of high dimensional search spaces associated with articulated body motion capture.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Kauai,Hawaii,2001,vol.2:669-676.
[35]陈睿,刘国翊,赵国英,张俊,李华.基于序列蒙特卡罗方法的3D人体运动跟踪.计算机辅助设计与图形学学报.2005,17(1):85-92.
[36]Lee M W,Cohen I,Jung S K.Particle filter with analytical inference for human body tracking.Workshop on Motion and Video Computing.Orlando,Florida,2002:159-165.
[37]胡士强.敬忠良.粒子滤波算法综述[J].控制与决策.2005.20(4):361-365.
[38]Haritaoglu I,Harwood D,Davis L.Multiple People Detection and Tracking Using Silhouettes[C].Fort Collins,Colorado:Proc 2nd IEEE Workshop on Visual Surveillance,1999.6-13.
[39]杜歆,陈建娟,王选贺.一种基于高斯概率模型的多人跟踪算法.传感技术学报.2009.已录用.
[40]Elgammal A E,Davis L S.Probabilistic framework for segmenting people under occlusion[C].IEEE International Conference on Computer Vision,2001,2:145-152.
[41]Prati A,Mikic I,Trivedi M M,Cucchiara R.Detecting moving shadows:formulation,algorithms and evaluation.IEEE Trans.Pattern Anal.Mach.Intell..2003,25(7):918-924.
[42]张莉.多摄像机人体跟踪技术的研究[博士学位论文].浙江大学.2008
[43]Koller D,Daniilidis K,Nagel H.Model-based object tracking in monocular image sequences of road traffic scenes.International Journal of Computer Vision.1993,10(3):257-281.
[44]Cucchiara R,Grana C,Piccardi M,Prati A.Detecting moving objects,ghosts and shadows in video streams.IEEE Transactions on Pattern Analysis and Machine Intelligence.2003,25(10):1337-1342.
[45]Horprasert T,Harwood D,Davis L S.A statistical approach for real-time robust background subtraction and shadow detection.Proceedings of the International Workshop on ICCV FRAME-RATE.Kerkyra,Greece,Sep.,1999:1-19.
[46]Xia S X,Li W C,Zhou Y.Improved k-means clustering algorithm[J].Journal of Southeast University(English Edition),2007,23(3),435-438.
[47]Ess,A.,B.Leibe,et al.(2008).A Mobile Vision System for Robust Multi-Person Tracking.Computer Vision and Pattern Recognition,2008.CVPR 2008.IEEE Conference on:1-8.
[48]Z.Zhang.Flexible Camera Calibration By Viewing a Plane From Unknown Orientations.International Conference on Computer Vision(ICCV'99),Corfu,Greece,Sep.,1999:666-673.
[49]Richard Hartley,Multiple View Geometry in Computer Vision(Second Edition) 2003.
[50]陈曦.陈建娟.谢立.一种基于RANSAC监视系统外参估算方法.哈尔滨工业大学学报.2009.已录用.
[51]Lowe D.Distinctive image features from scale-invariant keypoints.International Journal of Computer Vision.2004,60(2):91-110.
[52]Lindeberg Tony.Feature detection with automatic scale selection[J].International Journal of Computer Vision(S0920-5691),1998,30(2):79-116.
[53]Lowe,D.Distinctive image features from scale-invariant key points.International Journal of Computer Vision,2004,60(2):91-110.
[54]Martin A.Fischler and Robert C.Bolles(June 1981)."Random Sample Consensus:A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography".Comm.of the ACM 24:381-395.doi:10.1145/358669.358692.
[55]Javed O,Rasheed Z,Shafique K,Shah M.Tracking across multiple cameras with disjoint views.Proceedings of the Ninth IEEE International Conference on Computer Vision.2003,vol.2:952-957.
[56]Khan S,Shah M.Consistent labeling of tracked objects in multiple cameras with overlapping fields of view.IEEE Transactions on Pattern Analysis and Machine Intelligence.October 2003,25(10):1355-1360.
[57]Shi J,Tomashi C.Good feature to track.Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.USA,1994(2):593-600.
[58]Brown M,Burschka D,Hager G.Advances in computational stereo.IEEE Transactions on Pattern Analysis and Machine Intelligence.Aug,2003,25(8):993-1008.
[59]朱磊.一种基于直方图统计特征的直方图匹配算法的研究[J].计算技术与自动化,2004,23(2):48—51.