融合历史轨迹的智能汽车城市复杂环境多目标检测与跟踪算法
|
摘要
针对现有智能汽车环境感知算法多根据特定类型目标设计,在处理目标遮挡、光照突变等城市复杂场景时识别准确率较低的问题,提出一种基于网状分类器与融合历史轨迹的多目标检测与跟踪算法。该算法考虑各目标之间的遮挡关系,利用具有目标融合功能的网状分类器对多尺度滑动窗获取的待检窗口进行多目标检测;历史检测结果基于目标特征关联通过计算目标长短轨迹和历史轨迹可靠性验证生成历史轨迹库,该轨迹库用于预测或融合新的检测结果;利用该检测跟踪结果更新网状分类器中的标准差分类器、最近邻分类器和历史轨迹信息,直至完成多目标长时跟踪。实验结果表明,本文算法在目标遮挡、光照变化和阴雨天气的复杂城市环境下均可实现多目标长时间检测跟踪,与KITTI数据集样本相比,平均准确率在77.17%～81.32%之间,单帧图像平均耗时0.05s,具有较好的实时应用前景。
A new type of multi-target detection and tracking algorithm based on net classifiers and historical trajectories is proposed to focus the problem that most existing popular environmental perception algorithms for intelligent vehicles are designed for specific type of targets and they have been proved to be low recognition accuracy in complicated urban conditions where occlusion of different targets and abrupt light change are frequent.Images are scanned from a multi-scale sliding window and then fused by a fusion model according to occlusion relations among pedestrians,vehicles and traffic signs. Historical detection results are obtained based on correlation of target features through short trajectory generation,long trajectory generation and reliability verification of trajectories,and are used to generate a library of historical trajectories,which is used to predict and to fuse detection results in the next step.Then,the standard deviation classifier,the nearest neighbor classifier and historical trajectories are updated by new detection and tracking results. Multi-target long-time tracking in complicated conditions isrealized in this way.Field experiment shows that the proposed algorithm exhibits excellent performance of multi-target long-time tracking in complicated urban conditions such as rainy,shade,luminance changes.The average recognition rates for KITTI database samples is up to77.17%～81.32%and calculation time of single frame picture is only 0.05 s.These results show apromising algorithm for real-time application.
A new type of multi-target detection and tracking algorithm based on net classifiers and historical trajectories is proposed to focus the problem that most existing popular environmental perception algorithms for intelligent vehicles are designed for specific type of targets and they have been proved to be low recognition accuracy in complicated urban conditions where occlusion of different targets and abrupt light change are frequent.Images are scanned from a multi-scale sliding window and then fused by a fusion model according to occlusion relations among pedestrians,vehicles and traffic signs. Historical detection results are obtained based on correlation of target features through short trajectory generation,long trajectory generation and reliability verification of trajectories,and are used to generate a library of historical trajectories,which is used to predict and to fuse detection results in the next step.Then,the standard deviation classifier,the nearest neighbor classifier and historical trajectories are updated by new detection and tracking results. Multi-target long-time tracking in complicated conditions isrealized in this way.Field experiment shows that the proposed algorithm exhibits excellent performance of multi-target long-time tracking in complicated urban conditions such as rainy,shade,luminance changes.The average recognition rates for KITTI database samples is up to77.17%～81.32%and calculation time of single frame picture is only 0.05 s.These results show apromising algorithm for real-time application.
引文
[1]GAVRILA D.Pedestrian detection from a moving vehicle[C]∥Proceedings of the European Conference on Computer Vision.Berlin,Germany:Springer,2000:37-49.
[2]GAVRILA D M,GIEBEL J.Shape-based pedestrian detection and tracking[C]∥Proceedings of the Intelligent Vehicle Symposium.Piscataway,NJ,USA:IEEE,2002:8-14.
[3]SRINIVASA N.Vision-based vehicle detection and tracking method for forward collision warning in automobiles[C]∥Proceedings of the Intelligent Vehicle Symposium.Piscataway,NJ,USA:IEEE,2002:626-631.
[4]BERTOZZI M,BROGGI A,FASCIOLI A,et al.Stereo vision-based vehicle detection[C]∥Proceedings of the Intelligent Vehicles Symposium.Piscataway,NJ,USA:IEEE,2000:39-44.
[5]BROGGI A,CERRI P,ANTOELLO P C.Multiresolution vehicle detection using artificial vision[C]∥Proceedings of the Intelligent Vehicles Symposium.Piscataway,NJ,USA:IEEE,2004:310-314.
[6]ESTABLE S,SCHICK J,STEIN F,et al.A realtime traffic sign recognition system[C]∥Proceedings of the Intelligent Vehicles Symposium.Piscataway,NJ,USA,IEEE,1994:213-218.
[7]BERTOZZI M,BROGGI A,CASTELLUCCIO S.Areal-time oriented system for vehicle detection[J].Journal of Systems Architecture,1997,43(1):317-325.
[8]OUYANG W,WANG X.Joint deep learning for pedestrian detection[C]∥Proceedings of the International Conference on Computer Vision.Piscataway,NJ,USA:IEEE,2013:2056-2063.
[9]BETKE M,HARITAOGLU E,DAVIS L S.Multiple vehicle detection and tracking in hard real-time[C]∥Proceedings of the Intelligent Vehicles Symposium.Piscataway,NJ,USA:IEEE,1996:351-356.
[10]PAPAGEORGIOU C P,OREN M,POGGIO T.Ageneral framework for object detection[C]∥Proceedings of the International Conference on Computer Vision.Piscataway,NJ,USA:IEEE,1998:555-562.
[11]STALLKAMP J,SCHLIPSING M,SALMEN J,et al.Man vs.computer:benchmarking machine learning algorithms for traffic sign recognition[J].Neural Networks the Official Journal of the International Neural Network Society,2012,32(2):323-332.
[12]GIRSHICK R,DONAHUE J,DARRELL T,et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ,USA:IEEE,2014:580-587.
[13]GIRSHICK R.Fast R-CNN[C]∥Proceedings of the International Conference on Computer Vision.Piscataway,NJ,USA:IEEE,2015:1440-1448.
[14]REN Shaoqing,HE Kaiming,ROSS G,et al.Faster R-CNN:towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2015,39(6):1137-1149.
[15]REDMON J,DIVVALA S,GIRSHICK R,et al.You only look once:unified,real-time object detection[C]∥Proceedings of the Computer Vision and Pattern Recognition.Berlin,Germany:Springer,2015:779-788.
[16]REDMON J,FARHADI A.YOLO9000:better,faster,stronger[C]∥Proceedings of the Computer Vision and Pattern Recognition.Berlin,Germany:Springer,2017:21-25.
[17]LIU W,ANGUELOV D,ERHAN D,et al.SSD:single shot multibox detector[C]∥Proceedings of the European Conference on Computer Vision.Berlin,Germany:Springer,2016:21-37.
[18]JONATHAN L,EVAN S,TREVOR D.Fully convolutional networks for semantic segmentation[J].IEEETransactions on Pattern Analysis and Machine Intelligence,2017,4(39):640-651.
[19]PASZKE A,CHAURASIA A,KIM S,et al.ENet:a deep neural network architecture for real-time semantic segmentation[C]∥Proceedings of the Computer Vision and Pattern Recognition.Berlin,Germany:Springer,2017:21-25.
[20]ZHAO Hengshuang,SHI Jianping,QI Xiaojuan,et al.Pyramid scene parsing network[C]∥Proceedings of the Computer Vision and Pattern Recognition.Berlin,Germany:Springer,2017:21-25.
[21]ZHAO Hengshuang,QI Xiaojuan,SHEN Xiaoyong,et al.ICNet for real-time semantic segmentation on high-resolution images[C]∥Proceedings of the Computer Vision and Pattern Recognition.Berlin,Germany:Springer,2017:21-25.
[22]杨峰,王永齐,梁彦,等.基于概率假设密度滤波方法的多目标跟踪技术综述[J].自动化学报,2013,39(11):1944-1956.YANG Feng,WANG Yongqi,LIANG Yan,et al.Asurvey of PHD filter based multi-target tracking[J].ACTA Automatic Sinica,2013,39(11):1944-1956.
[23]OKUMA K,TALEGHANI A,FREITAS N D,et al.A boosted particle filter:multitarget detection and tracking[C]∥Proceedings of the European Conference on Computer Vision.Berlin,Germany:Springer,2004:28-39.
[24]ANDRIYENKO A,SCHINDLER K.Globally optimal multi-target tracking on a hexagonal lattice[C]∥Proceedings of the European Conference on Computer Vision.Berlin,Germany:Springer,2010:466-479.
[25]MALLICK M,KRANT J,SHALON J.Multi-sensor multi-target tracking with out-of-sequence measurements[C]∥Proceedings of the 6th International Conference of Information Fusion.Piscataway,NJ,USA:IEEE,2003:672-679.
[26]HOWARD A,PADGETT C,LIEBE C C.A multistage neural network for automatic target detection[C]∥Proceedings of the IEEE World Congress on Computational Intelligence.Piscataway,NJ,USA:IEEE,1998:4-9.
[27]GRINBERG M,OHR F,BEYERER J.Feature-based probabilistic data association(FBPDA)for visual multi-target detection and tracking under occlusions and split and merge effects[C]∥Proceedings of the International Conference on Intelligent Transportation Systems.Piscataway,NJ,USA:IEEE,2009:1-8.
[28]KALAL Z,MIKOLAJCZYK K,MATAS J.Tracking-learning-detection[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2011,34(7):1409-1422.
[29]HU Weiming,HU We,MAYBANK S.AdaBoostbased algorithm for network intrusion detection[J].IEEE Transactions on Systems Man and Cybernetics:Part B Cybernetics,2008,38(2):577-583.
[2]GAVRILA D M,GIEBEL J.Shape-based pedestrian detection and tracking[C]∥Proceedings of the Intelligent Vehicle Symposium.Piscataway,NJ,USA:IEEE,2002:8-14.
[3]SRINIVASA N.Vision-based vehicle detection and tracking method for forward collision warning in automobiles[C]∥Proceedings of the Intelligent Vehicle Symposium.Piscataway,NJ,USA:IEEE,2002:626-631.
[4]BERTOZZI M,BROGGI A,FASCIOLI A,et al.Stereo vision-based vehicle detection[C]∥Proceedings of the Intelligent Vehicles Symposium.Piscataway,NJ,USA:IEEE,2000:39-44.
[5]BROGGI A,CERRI P,ANTOELLO P C.Multiresolution vehicle detection using artificial vision[C]∥Proceedings of the Intelligent Vehicles Symposium.Piscataway,NJ,USA:IEEE,2004:310-314.
[6]ESTABLE S,SCHICK J,STEIN F,et al.A realtime traffic sign recognition system[C]∥Proceedings of the Intelligent Vehicles Symposium.Piscataway,NJ,USA,IEEE,1994:213-218.
[7]BERTOZZI M,BROGGI A,CASTELLUCCIO S.Areal-time oriented system for vehicle detection[J].Journal of Systems Architecture,1997,43(1):317-325.
[8]OUYANG W,WANG X.Joint deep learning for pedestrian detection[C]∥Proceedings of the International Conference on Computer Vision.Piscataway,NJ,USA:IEEE,2013:2056-2063.
[9]BETKE M,HARITAOGLU E,DAVIS L S.Multiple vehicle detection and tracking in hard real-time[C]∥Proceedings of the Intelligent Vehicles Symposium.Piscataway,NJ,USA:IEEE,1996:351-356.
[10]PAPAGEORGIOU C P,OREN M,POGGIO T.Ageneral framework for object detection[C]∥Proceedings of the International Conference on Computer Vision.Piscataway,NJ,USA:IEEE,1998:555-562.
[11]STALLKAMP J,SCHLIPSING M,SALMEN J,et al.Man vs.computer:benchmarking machine learning algorithms for traffic sign recognition[J].Neural Networks the Official Journal of the International Neural Network Society,2012,32(2):323-332.
[12]GIRSHICK R,DONAHUE J,DARRELL T,et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ,USA:IEEE,2014:580-587.
[13]GIRSHICK R.Fast R-CNN[C]∥Proceedings of the International Conference on Computer Vision.Piscataway,NJ,USA:IEEE,2015:1440-1448.
[14]REN Shaoqing,HE Kaiming,ROSS G,et al.Faster R-CNN:towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2015,39(6):1137-1149.
[15]REDMON J,DIVVALA S,GIRSHICK R,et al.You only look once:unified,real-time object detection[C]∥Proceedings of the Computer Vision and Pattern Recognition.Berlin,Germany:Springer,2015:779-788.
[16]REDMON J,FARHADI A.YOLO9000:better,faster,stronger[C]∥Proceedings of the Computer Vision and Pattern Recognition.Berlin,Germany:Springer,2017:21-25.
[17]LIU W,ANGUELOV D,ERHAN D,et al.SSD:single shot multibox detector[C]∥Proceedings of the European Conference on Computer Vision.Berlin,Germany:Springer,2016:21-37.
[18]JONATHAN L,EVAN S,TREVOR D.Fully convolutional networks for semantic segmentation[J].IEEETransactions on Pattern Analysis and Machine Intelligence,2017,4(39):640-651.
[19]PASZKE A,CHAURASIA A,KIM S,et al.ENet:a deep neural network architecture for real-time semantic segmentation[C]∥Proceedings of the Computer Vision and Pattern Recognition.Berlin,Germany:Springer,2017:21-25.
[20]ZHAO Hengshuang,SHI Jianping,QI Xiaojuan,et al.Pyramid scene parsing network[C]∥Proceedings of the Computer Vision and Pattern Recognition.Berlin,Germany:Springer,2017:21-25.
[21]ZHAO Hengshuang,QI Xiaojuan,SHEN Xiaoyong,et al.ICNet for real-time semantic segmentation on high-resolution images[C]∥Proceedings of the Computer Vision and Pattern Recognition.Berlin,Germany:Springer,2017:21-25.
[22]杨峰,王永齐,梁彦,等.基于概率假设密度滤波方法的多目标跟踪技术综述[J].自动化学报,2013,39(11):1944-1956.YANG Feng,WANG Yongqi,LIANG Yan,et al.Asurvey of PHD filter based multi-target tracking[J].ACTA Automatic Sinica,2013,39(11):1944-1956.
[23]OKUMA K,TALEGHANI A,FREITAS N D,et al.A boosted particle filter:multitarget detection and tracking[C]∥Proceedings of the European Conference on Computer Vision.Berlin,Germany:Springer,2004:28-39.
[24]ANDRIYENKO A,SCHINDLER K.Globally optimal multi-target tracking on a hexagonal lattice[C]∥Proceedings of the European Conference on Computer Vision.Berlin,Germany:Springer,2010:466-479.
[25]MALLICK M,KRANT J,SHALON J.Multi-sensor multi-target tracking with out-of-sequence measurements[C]∥Proceedings of the 6th International Conference of Information Fusion.Piscataway,NJ,USA:IEEE,2003:672-679.
[26]HOWARD A,PADGETT C,LIEBE C C.A multistage neural network for automatic target detection[C]∥Proceedings of the IEEE World Congress on Computational Intelligence.Piscataway,NJ,USA:IEEE,1998:4-9.
[27]GRINBERG M,OHR F,BEYERER J.Feature-based probabilistic data association(FBPDA)for visual multi-target detection and tracking under occlusions and split and merge effects[C]∥Proceedings of the International Conference on Intelligent Transportation Systems.Piscataway,NJ,USA:IEEE,2009:1-8.
[28]KALAL Z,MIKOLAJCZYK K,MATAS J.Tracking-learning-detection[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2011,34(7):1409-1422.
[29]HU Weiming,HU We,MAYBANK S.AdaBoostbased algorithm for network intrusion detection[J].IEEE Transactions on Systems Man and Cybernetics:Part B Cybernetics,2008,38(2):577-583.