基于三维分层图模型的复杂情况下多运动目标跟踪
|
摘要
复杂情况下的多运动目标跟踪是视频监控中的关键问题,在多目标运动中相互遮挡时二维视觉信息容易丢失而造成无法准确跟踪识别目标。本文采用kinect摄像机获取三维视觉信息,从节点、边、空间结构三个层次上建立目标的三维分层图模型表征目标的三维特征,并在每层进行通过视频帧间的匹配从而获得三维分层图模型匹配结果,并根据匹配结果先初步分析目标跟踪情况,如发生遮挡则通过遮挡区域聚类块与三维分层图模型中各特征匹配确定其匹配结果,从而得到多运动目标在复杂运动情况中的跟踪结果。实验表明,在实验室kinect拍摄的视频序列上当目标出现遮挡等复杂情况,也能取得较好的跟踪结果,在实验视频中比经典方法的跟踪总体性能指标改善约3%,说明本方法能较好地实现复杂情况下的多运动目标跟踪。
Tracking multiple moving objects under complex scenes is the key problem in visual surveillant system.When objects are occluded,two-dimensional information of objects can be lost easily,which will make the system can′t track and recognize objects.In this paper,the three-dimensional visual information is proposed by Kinect camera,which is used for building three-dimensional layered graph model form three different fields containing vertex,edge and spatial structure.The features of vertex,edge and spatial structure of three-dimensional layered graph model are matched between two consecutive frames separately,and the three-dimensional layered graph model are also matched by these vertex,edge and spatial structures.Then,the matching method is computed between consecutive frames by the three-dimensional layered models of different objects to track objects.When the objects are occluded,the clusters of the occlusion region are matched with the features of three-dimensional layered graph model to get the matching results,then the tracking results are determined even under occlusions.The experimental results show that the performance of proposed method is better than traditional method on the image sequences using Kinect camera even under occlusions,and the tracking performance rates are improved by 3%,which indicates that our method can track the multiple object better even under complex scenes.
Tracking multiple moving objects under complex scenes is the key problem in visual surveillant system.When objects are occluded,two-dimensional information of objects can be lost easily,which will make the system can′t track and recognize objects.In this paper,the three-dimensional visual information is proposed by Kinect camera,which is used for building three-dimensional layered graph model form three different fields containing vertex,edge and spatial structure.The features of vertex,edge and spatial structure of three-dimensional layered graph model are matched between two consecutive frames separately,and the three-dimensional layered graph model are also matched by these vertex,edge and spatial structures.Then,the matching method is computed between consecutive frames by the three-dimensional layered models of different objects to track objects.When the objects are occluded,the clusters of the occlusion region are matched with the features of three-dimensional layered graph model to get the matching results,then the tracking results are determined even under occlusions.The experimental results show that the performance of proposed method is better than traditional method on the image sequences using Kinect camera even under occlusions,and the tracking performance rates are improved by 3%,which indicates that our method can track the multiple object better even under complex scenes.
引文
[1] Huang Q,Xiang J,Hou J H,et al.Multi-target tracking algorithm based on feature fusion and discriminative appearance model[J].Journal of Image and Graphics,2015,20(9):1188-1198.黄奇,项俊,候建华,等.联合特征融合和判别性外观模型的多目标跟踪,中国图像图形学报,2015,20(9):1188-1198.
[2] TANG Chun-ming,LU Yong-wei.Pedestrian abnormal behavior analysis based on optimized sparse reconstruction algorithm[J].Computer Engineering and Applications.2017,53(8):165-169.汤春明,卢永伟.基于改进的稀疏重构算法的行人异常行为分析[J].计算机工程与应用,2017,53(8):165-169.
[3] LIU Ya-ting,WANG Kun-feng,WANG Fei-yue.Tracklet association-based visual object tracking:the state of the art and beyond[J].Acta Automatica Sinica,2017,43(11):1869-1885.刘雅婷,王坤峰,王飞跃.基于踪片Tracklet关联的视觉目标跟踪:现状与展望[J].自动化学报,2017,43(11):1869-1885.
[4] LU Hu-chuan,LI Pei-xia,WANG Dong.Visual object tracking:a survey[J].Pattern Recognition and Artificial Intelligence,2017,43(11):1869-1885.卢湖川,李佩霞,王栋.目标跟踪算法综述.模式识别与人工智能[J].2018,31(1):61-76.
[5] GUAN Hao,XUE Xiang-yang,AN Zhi-yong.Advances on application of deep learning for video object tacking[J].Acta Automatica Sinica,2016,42(6):834-847.管皓,薛向阳,安志勇.深度学习在视频目标跟踪中的应用进展与展望[J].自动化学报,2016,42(6):834-847.
[6] ZHOU Chun-luan,YUAN Jun-song.Bi-box regression for pedestrian detection and occlusion estimation.European Conference on Computer Vision (ECCV),2018,138-154.
[7] CHEN Bao-quan.SurveillanceaAnalysis system based on three-dimensional videos fusion,ITS 2013 Shenzhen international intelligent traffic exhibition,2013,5:26-28,Shenzhen Convention and Exhibition Center.陈宝权,基于三维视频融合的监控分析系统,ITS 2013深圳国际智能交通展览会,2013,5:26-28,深圳会展中心.
[8] ZHAO Ke,YU Hong-shan,WANG Yao-nan,et al.Improved 3D information visualization method for TOF camera based on vision projection[J].Journal of Electronic Measurement and Instrument,2012,26(6):529-534.赵科,余洪山,王耀南,等.基于视觉映射的三维相机信息可视化改善方法,电子测量与仪器学报,2012,26(6):529-534.
[9] LIU Jun,LIU Ye,ZHANG Gu-yue,et al.Detecting and tracking people in real time with RGB-D Camera.Pattern Recognition Letters,2015,53:16-23.
[10] ZHU Meng-zhe,FENG Rui.Detetion algorithm of pedestrian wandering based on 3D model[J].Computer Applications and Software,2017,34(4):149-155.朱梦哲,冯瑞.基于三维模型的行人徘徊行为检测算法[J].计算机应用与软件.2017,34(4):149-155.
[11] Rlza Alp Güler,Natalia Neverova,Iasonas Kokkinos.DensePose:dense human pose estimation in the wild.European Conference on Computer Vision(ECCV),2018,128-143.
[12] HU Meng-jie,LIU Zhen,ZHANG Jing-yu,et al.Robust object tracking via multi-cue fusion.Signal Processing,2017,139:86-95.
[13] TAN Yan,WANG Yu-jun,LI Fei-long,et al.On moving objects tracking and 3D measurement based on kinect.[J]Journal of Southwest China Normal University(Natural Science Edition),2013,38(9):101-105.谭艳,王宇俊,李飞龙,等.基于Kinect的运动目标跟踪与三维测量[J].西南师范大学学报(自然科学版),2013,38(9):101-105.
[14] José Guedes dos Santos Júnior,Joo Paulo Silva do Monte Lima.Particle swarm optimization for 3D object tracking in RGB-D images.Special Section on SVR,2018:167-180.
[15] Jungong Han,Eric J.Pauwels,et al.de With,Employing a RGB-D sensor for real-time tracking of humans across multiple re-entries in a smart environment,IEEE Trans.On Consumer Electronics,2014,58(2):255-263.
[16] ZHANG Hao,Christopher Reardon,Lynne E.Real-time multiple human perception with color-depth cameras on a mobile robot,IEEE trans on Cybernetics,2013,43(5):1429-1440.
[17] ZHAO Yang,LIU Zi-cheng,CHENG Hong.RGB-depth feature for 3D human activity recognition,China Communications,2013,7:93-103.
[18] GAO Shan,HAN Zhen-jun,LI Ce-li,et al.Real-time multipedestrian tracking in traffic scenesvia an RGB-D-based layered graph model,IEEE trans onIntelligent Transportation Systems,2015,16(5):2814-2825.
[19] YANG Han-fang,ZHANG Guo-shan,WANG Xin-bo,et al.Human object tracking in depth image based on Snake algorithm[J].Journal of TianJin University of Technology,2014,30(5):41-45.杨晗芳,张国山,王欣博,等.基于Snake算法的深度图像人体目标跟踪,天津理工大学学报,2014,30(5):41-45.
[20] Amit Kumar K C,Laurent Jacques,Christophe De Vleeschouwer.Discriminative and efficient label propagation on complementary graphs for multi-object tracking.IEEE transactions on Pattern Analysis and Machine Intelligence,2017,39(1):61-74.
[21] JIANG Ming-xin,QIU Tian-shuang,CHI Hong-bo.A novel multrobject tracking algorithm based on RJMCMC with RGB and depth information fusion[J].Journal of Optoelectronics·Laser,2015,26(7):1342-1348.姜明新,邱天爽,迟宏波.基于颜色与深度信息特征融合的一种多目标跟踪新算法[J].光电子·激光,2015,26(7):1342-1348.
[2] TANG Chun-ming,LU Yong-wei.Pedestrian abnormal behavior analysis based on optimized sparse reconstruction algorithm[J].Computer Engineering and Applications.2017,53(8):165-169.汤春明,卢永伟.基于改进的稀疏重构算法的行人异常行为分析[J].计算机工程与应用,2017,53(8):165-169.
[3] LIU Ya-ting,WANG Kun-feng,WANG Fei-yue.Tracklet association-based visual object tracking:the state of the art and beyond[J].Acta Automatica Sinica,2017,43(11):1869-1885.刘雅婷,王坤峰,王飞跃.基于踪片Tracklet关联的视觉目标跟踪:现状与展望[J].自动化学报,2017,43(11):1869-1885.
[4] LU Hu-chuan,LI Pei-xia,WANG Dong.Visual object tracking:a survey[J].Pattern Recognition and Artificial Intelligence,2017,43(11):1869-1885.卢湖川,李佩霞,王栋.目标跟踪算法综述.模式识别与人工智能[J].2018,31(1):61-76.
[5] GUAN Hao,XUE Xiang-yang,AN Zhi-yong.Advances on application of deep learning for video object tacking[J].Acta Automatica Sinica,2016,42(6):834-847.管皓,薛向阳,安志勇.深度学习在视频目标跟踪中的应用进展与展望[J].自动化学报,2016,42(6):834-847.
[6] ZHOU Chun-luan,YUAN Jun-song.Bi-box regression for pedestrian detection and occlusion estimation.European Conference on Computer Vision (ECCV),2018,138-154.
[7] CHEN Bao-quan.SurveillanceaAnalysis system based on three-dimensional videos fusion,ITS 2013 Shenzhen international intelligent traffic exhibition,2013,5:26-28,Shenzhen Convention and Exhibition Center.陈宝权,基于三维视频融合的监控分析系统,ITS 2013深圳国际智能交通展览会,2013,5:26-28,深圳会展中心.
[8] ZHAO Ke,YU Hong-shan,WANG Yao-nan,et al.Improved 3D information visualization method for TOF camera based on vision projection[J].Journal of Electronic Measurement and Instrument,2012,26(6):529-534.赵科,余洪山,王耀南,等.基于视觉映射的三维相机信息可视化改善方法,电子测量与仪器学报,2012,26(6):529-534.
[9] LIU Jun,LIU Ye,ZHANG Gu-yue,et al.Detecting and tracking people in real time with RGB-D Camera.Pattern Recognition Letters,2015,53:16-23.
[10] ZHU Meng-zhe,FENG Rui.Detetion algorithm of pedestrian wandering based on 3D model[J].Computer Applications and Software,2017,34(4):149-155.朱梦哲,冯瑞.基于三维模型的行人徘徊行为检测算法[J].计算机应用与软件.2017,34(4):149-155.
[11] Rlza Alp Güler,Natalia Neverova,Iasonas Kokkinos.DensePose:dense human pose estimation in the wild.European Conference on Computer Vision(ECCV),2018,128-143.
[12] HU Meng-jie,LIU Zhen,ZHANG Jing-yu,et al.Robust object tracking via multi-cue fusion.Signal Processing,2017,139:86-95.
[13] TAN Yan,WANG Yu-jun,LI Fei-long,et al.On moving objects tracking and 3D measurement based on kinect.[J]Journal of Southwest China Normal University(Natural Science Edition),2013,38(9):101-105.谭艳,王宇俊,李飞龙,等.基于Kinect的运动目标跟踪与三维测量[J].西南师范大学学报(自然科学版),2013,38(9):101-105.
[14] José Guedes dos Santos Júnior,Joo Paulo Silva do Monte Lima.Particle swarm optimization for 3D object tracking in RGB-D images.Special Section on SVR,2018:167-180.
[15] Jungong Han,Eric J.Pauwels,et al.de With,Employing a RGB-D sensor for real-time tracking of humans across multiple re-entries in a smart environment,IEEE Trans.On Consumer Electronics,2014,58(2):255-263.
[16] ZHANG Hao,Christopher Reardon,Lynne E.Real-time multiple human perception with color-depth cameras on a mobile robot,IEEE trans on Cybernetics,2013,43(5):1429-1440.
[17] ZHAO Yang,LIU Zi-cheng,CHENG Hong.RGB-depth feature for 3D human activity recognition,China Communications,2013,7:93-103.
[18] GAO Shan,HAN Zhen-jun,LI Ce-li,et al.Real-time multipedestrian tracking in traffic scenesvia an RGB-D-based layered graph model,IEEE trans onIntelligent Transportation Systems,2015,16(5):2814-2825.
[19] YANG Han-fang,ZHANG Guo-shan,WANG Xin-bo,et al.Human object tracking in depth image based on Snake algorithm[J].Journal of TianJin University of Technology,2014,30(5):41-45.杨晗芳,张国山,王欣博,等.基于Snake算法的深度图像人体目标跟踪,天津理工大学学报,2014,30(5):41-45.
[20] Amit Kumar K C,Laurent Jacques,Christophe De Vleeschouwer.Discriminative and efficient label propagation on complementary graphs for multi-object tracking.IEEE transactions on Pattern Analysis and Machine Intelligence,2017,39(1):61-74.
[21] JIANG Ming-xin,QIU Tian-shuang,CHI Hong-bo.A novel multrobject tracking algorithm based on RJMCMC with RGB and depth information fusion[J].Journal of Optoelectronics·Laser,2015,26(7):1342-1348.姜明新,邱天爽,迟宏波.基于颜色与深度信息特征融合的一种多目标跟踪新算法[J].光电子·激光,2015,26(7):1342-1348.