基于深度学习的红外与可见光决策级融合跟踪
|
摘要
提出了一种基于深度学习的红外与可见光决策级融合跟踪方法。通过建立参数传递模型,从现有基于深度学习的检测模型中抽取指定对象的可见光检测模型,作为红外检测的预训练模型,在采集的红外图像数据集上进行微调训练,得到基于深度学习的红外检测模型。在此基础上,建立了基于深度学习的红外与可见光决策级融合跟踪模型,进行了单波段跟踪与双波段融合跟踪对比实验。结果表明,所提方法跟踪精度和成功率比单波段跟踪均有所提升,具有较好的稳健性。
A decision-level fusion tracking method based on deep learning for infrared and visible spectra is proposed.By building the parameter transfer model,the visible detection model of the specified objects is extracted from the existing deep-learning-based detection model.This visible detection model is used as the infrared detection pre-training model,and the fine-tuning training on a collected infrared image dataset is done to obtain the infrared detection model based on deep learning.On this basis,a decision-level fusion tracking model based on deep learning is built.An comparison experiment between single-band tracking and dual-band fusion tracking is carried out.The research results show that the proposed method improves the tracking accuracy and success rate compared with the single-band tracking,and has good robustness.
A decision-level fusion tracking method based on deep learning for infrared and visible spectra is proposed.By building the parameter transfer model,the visible detection model of the specified objects is extracted from the existing deep-learning-based detection model.This visible detection model is used as the infrared detection pre-training model,and the fine-tuning training on a collected infrared image dataset is done to obtain the infrared detection model based on deep learning.On this basis,a decision-level fusion tracking model based on deep learning is built.An comparison experiment between single-band tracking and dual-band fusion tracking is carried out.The research results show that the proposed method improves the tracking accuracy and success rate compared with the single-band tracking,and has good robustness.
引文
[1]Sivanantham S,Paul N N,Iyer R S.Object tracking algorithm implementation for security applications[J].Far East Journal of Electronics and Communications,2016,16(1):1-13.
[2]Tang C,Ling Y S,Zheng K D,et al.Object detection method of multi-view SSD based on deep learning[J].Infrared and Laser Engineering,2018,47(1):0126003.唐聪,凌永顺,郑科栋,等.基于深度学习的多视窗SSD目标检测方法[J].红外与激光工程,2018,47(1):0126003.
[3]Luo H B,Xu L Y,Hui B,et al.Status and prospect of target tracking based on deep learning[J].Infrared and Laser Engineering,2017,46(5):0502002.罗海波,许凌云,惠斌,等.基于深度学习的目标跟踪方法研究现状与展望[J].红外与激光工程,2017,46(5):0502002.
[4]Hall D L,Llinas J.An introduction to multisensor data fusion[J].Proceedings of the IEEE,1997,85(1):6-23.
[5]Liu H P,Sun F C.Fusion tracking in color and infrared images using joint sparse representation[J].Science China Information Sciences,2012,55(3):590-599.
[6]Zhao G P,Bo Y M,Chen Y.A method for tracking object in infrared and visible image based on multiple features[J].Acta Armamentarii,2011,32(4):445-451.赵高鹏,薄煜明,陈益.多特征提取的红外和可见光目标跟踪方法[J].兵工学报,2011,32(4):445-451.
[7]Yun X,Jing Z L,Xiao G,et al.A compressive tracking based on time-space Kalman fusion model[J].Science China Information Sciences,2016,59(1):012106.
[8]Wang N Y,Yeung D Y.Learning a deep compact image representation for visual tracking[C]∥International Conference on Neural Information Processing Systems,2013:809-817.
[9]Ma C,Huang J B,Yang X K,et al.Hierarchical convolutional features for visual tracking[C]∥IEEEInternational Conference on Computer Vision,2015:3074-3082.
[10]Nam H,Han B.Learning multi-domain convolutional neural networks for visual tracking[C]∥IEEEConference on Computer Vision and Pattern Recognition,2016:4293-4302.
[11]Tang C,Ling Y S,Yang H,et al.A visual tracking method via object detection based on deep learning[J].Infrared and Laser Engineering,2018,47(5):0526001.唐聪,凌永顺,杨华,等.基于深度学习物体检测的视觉跟踪方法[J].红外与激光工程,2018,47(5):0526001.
[12]Fendri E,Boukhriss R R,Hammami M.Fusion of thermal infrared and visible spectra for robust moving object detection[J].Pattern Analysis and Applications,2017,20(4):907-926.
[13]Guo Y M,Liu Y,Oerlemans A,et al.Deep learning for visual understanding:a review[J].Neurocomputing,2016,187:27-48.
[14]Wang L,Liu T,Wang G,et al.Video tracking using learned hierarchical features[J].IEEETransactions on Image Processing,2015,24(4):1424-1435.
[15]Wang N,Li S,Gupta A,et al.Transferring rich feature hierarchies for robust visual tracking[J].CarX iv preprint arX iv:1501.04587,,2015.
[16]Wang X,Hou Z Q,Yu W S,et al.Robust visual tracking via multiscale deep sparse networks[J].Optical Engineering,2017,56(4):043107.
[17]Russakovsky O,Deng J,Su H,et al.ImageNet large scale visual recognition challenge[J].International Journal of Computer Vision,2015,115(3):211-252.
[18]Vicente S,Carreira J,Agapito L,et al.Reconstructing PASCAL VOC[C]∥IEEEConference on Computer Vision and Pattern Recognition,2014:41-48.
[19]Liu W,Anguelov D,Erhan D,et al.SSD:single shot multibox detector[C]∥European Conference on Computer Vision,2016:21-37.
[20]Davis J,Goadrich M.The relationship between precision-recall and ROC curves[C]∥International Conference on Machine Learning,2006:233-240.
[21]Wu Y,Lim J,Yang M H.Object tracking benchmark[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(9):1834-1848.
[2]Tang C,Ling Y S,Zheng K D,et al.Object detection method of multi-view SSD based on deep learning[J].Infrared and Laser Engineering,2018,47(1):0126003.唐聪,凌永顺,郑科栋,等.基于深度学习的多视窗SSD目标检测方法[J].红外与激光工程,2018,47(1):0126003.
[3]Luo H B,Xu L Y,Hui B,et al.Status and prospect of target tracking based on deep learning[J].Infrared and Laser Engineering,2017,46(5):0502002.罗海波,许凌云,惠斌,等.基于深度学习的目标跟踪方法研究现状与展望[J].红外与激光工程,2017,46(5):0502002.
[4]Hall D L,Llinas J.An introduction to multisensor data fusion[J].Proceedings of the IEEE,1997,85(1):6-23.
[5]Liu H P,Sun F C.Fusion tracking in color and infrared images using joint sparse representation[J].Science China Information Sciences,2012,55(3):590-599.
[6]Zhao G P,Bo Y M,Chen Y.A method for tracking object in infrared and visible image based on multiple features[J].Acta Armamentarii,2011,32(4):445-451.赵高鹏,薄煜明,陈益.多特征提取的红外和可见光目标跟踪方法[J].兵工学报,2011,32(4):445-451.
[7]Yun X,Jing Z L,Xiao G,et al.A compressive tracking based on time-space Kalman fusion model[J].Science China Information Sciences,2016,59(1):012106.
[8]Wang N Y,Yeung D Y.Learning a deep compact image representation for visual tracking[C]∥International Conference on Neural Information Processing Systems,2013:809-817.
[9]Ma C,Huang J B,Yang X K,et al.Hierarchical convolutional features for visual tracking[C]∥IEEEInternational Conference on Computer Vision,2015:3074-3082.
[10]Nam H,Han B.Learning multi-domain convolutional neural networks for visual tracking[C]∥IEEEConference on Computer Vision and Pattern Recognition,2016:4293-4302.
[11]Tang C,Ling Y S,Yang H,et al.A visual tracking method via object detection based on deep learning[J].Infrared and Laser Engineering,2018,47(5):0526001.唐聪,凌永顺,杨华,等.基于深度学习物体检测的视觉跟踪方法[J].红外与激光工程,2018,47(5):0526001.
[12]Fendri E,Boukhriss R R,Hammami M.Fusion of thermal infrared and visible spectra for robust moving object detection[J].Pattern Analysis and Applications,2017,20(4):907-926.
[13]Guo Y M,Liu Y,Oerlemans A,et al.Deep learning for visual understanding:a review[J].Neurocomputing,2016,187:27-48.
[14]Wang L,Liu T,Wang G,et al.Video tracking using learned hierarchical features[J].IEEETransactions on Image Processing,2015,24(4):1424-1435.
[15]Wang N,Li S,Gupta A,et al.Transferring rich feature hierarchies for robust visual tracking[J].CarX iv preprint arX iv:1501.04587,,2015.
[16]Wang X,Hou Z Q,Yu W S,et al.Robust visual tracking via multiscale deep sparse networks[J].Optical Engineering,2017,56(4):043107.
[17]Russakovsky O,Deng J,Su H,et al.ImageNet large scale visual recognition challenge[J].International Journal of Computer Vision,2015,115(3):211-252.
[18]Vicente S,Carreira J,Agapito L,et al.Reconstructing PASCAL VOC[C]∥IEEEConference on Computer Vision and Pattern Recognition,2014:41-48.
[19]Liu W,Anguelov D,Erhan D,et al.SSD:single shot multibox detector[C]∥European Conference on Computer Vision,2016:21-37.
[20]Davis J,Goadrich M.The relationship between precision-recall and ROC curves[C]∥International Conference on Machine Learning,2006:233-240.
[21]Wu Y,Lim J,Yang M H.Object tracking benchmark[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(9):1834-1848.