背景与时间感知的相关滤波实时视觉跟踪
|
摘要
目的传统的相关滤波跟踪算法采用对跟踪目标(唯一准确正样本)循环移位获取负样本,在整个学习过程中没有对真正的背景信息进行建模,因此当目标与背景信息极其相似时容易漂移。大多数跟踪算法为了提高跟踪性能,在时间序列上收集了大量的训练样本而导致计算复杂度的增加。采用模型在线更新策略,由于未考虑时间一致性,使得学习到的滤波器可能偏向背景而发生漂移。为了改善以上问题,本文在背景感知相关滤波(BACF)跟踪算法的基础上,加入时间感知,构建了一个带等式限制的相关滤波目标函数,称为背景与时间感知相关滤波(BTCF)视觉跟踪。该算法不但获取了真正的负样本作为训练集,而且仅用当前帧信息无需模型在线更新策略就能学习到具有较强判别力的相关滤波器。方法首先将带等式限制的相关滤波目标函数转化为无约束的增广拉格朗日乘子公式,然后采用交替方向乘子方法(ADMM)转化为两个具有闭式解的子问题迭代求最优解。结果采用OTB2015数据库中的OPE(one pass evaluation)评价准则,以成功率曲线图线下面积(AUC)和中心点位置误差为评判标准,在OTB2015公开数据库上与10个比较优秀的视觉跟踪算法进行对比实验。结果显示,100个视频序列和11个视频属性的成功率及对应的AUC和中心位置误差均明显优于其他基于相关滤波的视觉跟踪算法,说明本文算法具有良好的跟踪效果。本文的BTCF算法仅采用HOG纯手工特征,在OTB2015数据库上AUC较BACF算法提高了1. 3%;由于颜色与边缘特征具有互补特性,本文融合CN(color names)特征后,在OTB2015数据库上,AUC较BACF算法提高了4. 2%,采用纯手工特征跟踪性能AUC达到0. 663,跟踪速度达到25. 4帧/s。结论本文的BTCF算法能够适用于光照变化、目标旋转、遮挡等复杂情况下的视觉跟踪,具有良好的鲁棒性和一定的实时性。
Objective Visual tracking is a classical computer vision problem with many applications. In generic visual tracking,the task is to estimate the trajectory of a target in an image sequence,given only its initial location. Recently,traditional discriminative correlation filter-based approaches have been successfully applied to tracking problems. These methods learn a discriminative correlation filter from a set of training samples,which adopt a circular shift operator on the tracking target object( the only accurate positive sample) to obtain the training negative samples. These shifted patches are implicitly generated through the circulatory property of correlation in frequency domain and are used as negative examples for training the filter. All shifted patches are plagued by circular boundary effects and are not truly representative of negative patches in real-word scenes. Thus,the actual background information is not modeled during the total learning process and when the target object is similar to the background information,thereby leading to a drift. To improve the performance,a large number of training samples are collected,which results in the increase of computational complexity. Moreover,preferring the background is easy due to the online model update strategy,which causes drift. To resolve this problem,we construct a discriminative correlation filter-based target function with equation-constrained condition on the background-aware correlation filtering( BACF) visual object tracking algorithm,which is termed as the background-temporal-aware correlation filter( BTCF) visual object tracking. Our algorithm obtains the actual negative sample with the same size as the target object on the training set by multiplying the filter with a binary mask to suppress the background region. Moreover,it can learn a strong correlation filter-based discriminative classifier by only using the current frame information without online updating of the model. Method In this paper,the proposed BTCF model is convex and can be minimized to obtain the globally optimal solution. In order to further reduce the computational burden,we propose a new equation-constrained discriminative correlation filter-based objective function. This objective function satisfies the Eckstein-Bertsekas condition,therefore,it can be transformed into an unconstrained augmented Lagrange multiplier formula to converge to the global optimum solution. Then,two sub-problems with closed-form solution are gained by using the alternating direction multiplier method(ADMM). Every sub-problem is a smooth and convex function and is very easy to obtain the solution,therefore,each iteration of the sub-problem has a closed-form solution and is the global optimal solution of each sub-problem. Because of the convolution calculation in sub-problem two,it is difficult to solve the optimization problem,consequently,according to Parsevals ' theorem,we transform sub-problem two into Fourier domain to reduce computational complexity. The efficient ADMM based approach for learning our filter on multi-channel features,with computational cost of O( LKTlg( T)),where T is the size of vectorized frame,K is the number of feature channels,and L is the ADMMs' iterations. We calculate model updates with Sherman-Morrison lemma to cope with changes in target and background appearance with real-time performance. Our algorithm can empirically converge within few iterations and,with hand-crafted features,can run in real time,thereby achieving notable improvements over BACF object tracking algorithm by tracking accuracy. Result The one-pass evaluation is used to compare different trackers proposed by OTB2015 based on two criteria,namely,center location error and bounding box overlap ratio. The center location error is one of the widely used evaluation metrics for target object tracking,which computes the average Euclidean distance between the center locations of the tracked targets and manually labeled ground truth positions of all the frames. Moreover,the percentage of frames in the estimated locations with a given threshold distance of the ground truth positions is considered successful tracking. Another commonly used evaluation metric is the overlap score. We use the area under the curve( AUC) of each success plot,which is the average of the success rates corresponding to the sample over thresholds to measure the ranking trackers. Our approach is compared with 10 stateof-the-art visual object tracking algorithms on the OTB2015 public database. Results show that our BTCF algorithm is remarkably better than visual tracking algorithms based on discriminative correlation filter in center location error and AUC.OTB2015 categorizes 100 sequences by annotating them with 11 attributes to evaluate and analyze the strength and weakness of the tracking approaches. Results show that our BTCF algorithm is remarkably better than the BACF in the center location error and AUC on the 11 attributes,thereby indicating that our algorithm can achieve effective and efficient performance.The BTCF visual object tracking increased 1. 3% on AUC compared with BACF,which only uses histogram of oriented gradients( HOG) hand-crafted features on the OTB2015 database. The color and edge features have complementary characteristics; thus,we introduce color names( CNs) to our BTCF formulation to hoist the 4. 2% AUC compared with BACF on the OTB2015 database. The AUC that reaches 0. 663 and the speed that attains 25. 4 fps on OTB2015 database only use the hand-crafted features( HOG and CNs). Conclusion Compared with the BACF algorithm and other current popular tracking approaches,the proposed BTCF-based visual tracking algorithm can be applied to many challenging conditions. Due to introduce the temporal-aware term on BACF model,a stronger discriminative classifier can be learned to separate the target from the background,especially in illumination variation,motion blur,out-of-plane rotation,and occlusion scene. Therefore,the proposed BTCF-based algorithm demonstrates the robustness and real-time characteristic.
Objective Visual tracking is a classical computer vision problem with many applications. In generic visual tracking,the task is to estimate the trajectory of a target in an image sequence,given only its initial location. Recently,traditional discriminative correlation filter-based approaches have been successfully applied to tracking problems. These methods learn a discriminative correlation filter from a set of training samples,which adopt a circular shift operator on the tracking target object( the only accurate positive sample) to obtain the training negative samples. These shifted patches are implicitly generated through the circulatory property of correlation in frequency domain and are used as negative examples for training the filter. All shifted patches are plagued by circular boundary effects and are not truly representative of negative patches in real-word scenes. Thus,the actual background information is not modeled during the total learning process and when the target object is similar to the background information,thereby leading to a drift. To improve the performance,a large number of training samples are collected,which results in the increase of computational complexity. Moreover,preferring the background is easy due to the online model update strategy,which causes drift. To resolve this problem,we construct a discriminative correlation filter-based target function with equation-constrained condition on the background-aware correlation filtering( BACF) visual object tracking algorithm,which is termed as the background-temporal-aware correlation filter( BTCF) visual object tracking. Our algorithm obtains the actual negative sample with the same size as the target object on the training set by multiplying the filter with a binary mask to suppress the background region. Moreover,it can learn a strong correlation filter-based discriminative classifier by only using the current frame information without online updating of the model. Method In this paper,the proposed BTCF model is convex and can be minimized to obtain the globally optimal solution. In order to further reduce the computational burden,we propose a new equation-constrained discriminative correlation filter-based objective function. This objective function satisfies the Eckstein-Bertsekas condition,therefore,it can be transformed into an unconstrained augmented Lagrange multiplier formula to converge to the global optimum solution. Then,two sub-problems with closed-form solution are gained by using the alternating direction multiplier method(ADMM). Every sub-problem is a smooth and convex function and is very easy to obtain the solution,therefore,each iteration of the sub-problem has a closed-form solution and is the global optimal solution of each sub-problem. Because of the convolution calculation in sub-problem two,it is difficult to solve the optimization problem,consequently,according to Parsevals ' theorem,we transform sub-problem two into Fourier domain to reduce computational complexity. The efficient ADMM based approach for learning our filter on multi-channel features,with computational cost of O( LKTlg( T)),where T is the size of vectorized frame,K is the number of feature channels,and L is the ADMMs' iterations. We calculate model updates with Sherman-Morrison lemma to cope with changes in target and background appearance with real-time performance. Our algorithm can empirically converge within few iterations and,with hand-crafted features,can run in real time,thereby achieving notable improvements over BACF object tracking algorithm by tracking accuracy. Result The one-pass evaluation is used to compare different trackers proposed by OTB2015 based on two criteria,namely,center location error and bounding box overlap ratio. The center location error is one of the widely used evaluation metrics for target object tracking,which computes the average Euclidean distance between the center locations of the tracked targets and manually labeled ground truth positions of all the frames. Moreover,the percentage of frames in the estimated locations with a given threshold distance of the ground truth positions is considered successful tracking. Another commonly used evaluation metric is the overlap score. We use the area under the curve( AUC) of each success plot,which is the average of the success rates corresponding to the sample over thresholds to measure the ranking trackers. Our approach is compared with 10 stateof-the-art visual object tracking algorithms on the OTB2015 public database. Results show that our BTCF algorithm is remarkably better than visual tracking algorithms based on discriminative correlation filter in center location error and AUC.OTB2015 categorizes 100 sequences by annotating them with 11 attributes to evaluate and analyze the strength and weakness of the tracking approaches. Results show that our BTCF algorithm is remarkably better than the BACF in the center location error and AUC on the 11 attributes,thereby indicating that our algorithm can achieve effective and efficient performance.The BTCF visual object tracking increased 1. 3% on AUC compared with BACF,which only uses histogram of oriented gradients( HOG) hand-crafted features on the OTB2015 database. The color and edge features have complementary characteristics; thus,we introduce color names( CNs) to our BTCF formulation to hoist the 4. 2% AUC compared with BACF on the OTB2015 database. The AUC that reaches 0. 663 and the speed that attains 25. 4 fps on OTB2015 database only use the hand-crafted features( HOG and CNs). Conclusion Compared with the BACF algorithm and other current popular tracking approaches,the proposed BTCF-based visual tracking algorithm can be applied to many challenging conditions. Due to introduce the temporal-aware term on BACF model,a stronger discriminative classifier can be learned to separate the target from the background,especially in illumination variation,motion blur,out-of-plane rotation,and occlusion scene. Therefore,the proposed BTCF-based algorithm demonstrates the robustness and real-time characteristic.
引文
[1]Bolme D S,Beveridge J R,Draper B A,et al.Visual object tracking using adaptive correlation filters[C]//Proceedings of2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.San Francisco,CA,USA:IEEE,2010:2544-2550.[DOI:10.1109/CVPR.2010.5539960]
[2]Henriques J F,Caseiro R,Martins P,et al.Exploiting the circulant structure of tracking-by-detection with kernels[C]//Proceedings of the 12th European Conference on Computer Vision.Florence,Italy:Springer,2012:702-715.[DOI:10.1007/978-3-642-33765-9 50]
[3]Danelljan M,Khan F S,Felsberg M,et al.Adaptive color attributes for real-time visual tracking[C]//Proceedings of 2014IEEE Conference on Computer Vision and Pattern Recognition.Columbus,OH,USA:IEEE,2014:1090-1097.[DOI:10.1109/CVPR.2014.143]
[4]Henriques J F,Caseiro R,Martins P,et al.High-speed tracking with kernelized correlation filters[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(3):583-596.[DOI:10.1109/TPAMI.2014.2345390]
[5]Tang M,Feng J Y.Multi-kernel correlation filter for visual tracking[C]//Proceedings of 2015 IEEE International Conference on Computer Vision.Santiago,Chile:IEEE,2015:3038-3046.[DOI:10.1109/ICCV.2015.348]
[6]Li Y,Zhu J K.A scale adaptive kernel correlation filter tracker with feature integration[C]//Proceedings of European Conference on Computer Vision.Zurich,Switzerland:Springer,2015:254-265.[DOI:10.1007/978-3-319-16181-5_18]
[7]Danelljan M,Hger G,Khan F S,et al.Accurate scale estimation for robust visual tracking[C]//Proceedings of British Machine Vision Conference.Nottingham,UK:BMVA Press,2014:65.1-65.11.[DOI:10.5244/C.28.65]
[8]Danelljan M,Hger G,Khan F S,et al.Discriminative scale space tracking.[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(8):1561-1575.[DOI:10.1109/TPAMI.2016.2609928]
[9]Galoogahi H K,Sim T,Lucey S.Correlation filters with limited boundaries[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston,MA,USA:IEEE,2015:4630-4638.[DOI:10.1109/CVPR.2015.7299094]
[10]Danelljan M,Hger G,Khan F S,et al.Learning spatially regularized correlation filters for visual tracking[C]//Proceedings of2015 IEEE International Conference on Computer Vision.Santiago,Chile:IEEE,2015:4310-4318.[DOI:10.1109/ICCV.2015.490]
[11]Danelljan M,Robinson A,Khan F S,et al.Beyond correlation filters:learning continuous convolution operators for visual tracking[C]//Proceedings of the 14th European Conference on Computer Vision.Amsterdam,Netherlands:Springer,2016:472-488.[DOI:10.1007/978-3-319-46454-1_29]
[12]Danelljan M,Bhat G,Khan F S,et al.ECO:efficient convolution operators for tracking[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,HI,USA:IEEE,2017:6931-6939.[DOI:10.1109/CVPR.2017.733]
[13]Nam H,Han B.Learning multi-domain convolutional neural networks for visual tracking[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,NV,USA:IEEE,2016:4293-4302.[DOI:10.1109/CVPR.2016.465]
[14]Song Y B,Ma C,Gong L J,et al.CREST:convolutional residual learning for visual tracking[C]//Proceedings of 2017 IEEEInternational Conference on Computer Vision.Venice,Italy:IEEE,2017:2574-2583.[DOI:10.1109/ICCV.2017.279]
[15]Gundogdu E,Alatan A A.Good features to correlate for visual tracking[J].IEEE Transactions on Image Processing,2018,27(5):2526-2540.[DOI:10.1109/TIP.2018.2806280]
[16]Danelljan M,Hger G,Khan F S,et al.Convolutional features for correlation filter based visual tracking[C]//Proceedings of2015 IEEE International Conference on Computer Vision Workshop.Santiago,Chile:IEEE,2015:621-629.[DOI:10.1109/ICCVW.2015.84]
[17]Galoogahi H K,Fagg A,Lucey S.Learning background-aware correlation filters for visual tracking[C]//Proceedings of 2017IEEE International Conference on Computer Vision.Venice,Italy:IEEE,2017:1144-1152.[DOI:10.1109/ICCV.2017.129]
[18]Wu Y,Lim J,Yang M H.Object tracking benchmark[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(9):1834-1848.[DOI:10.1109/TPAMI.2014.2388226]
[19]Zhang W,Kang B S.Recent advances in correlation filter-based object tracking:a review[J].Journal of Image and Graphics,2017,22(8):1017-1033.[张微,康宝生.相关滤波目标跟踪进展综述[J].中国图象图形学报,2017,22(8):1017-1033.][DOI:10.11834/jig.170092]
[20]Lu H C,Li P X,Wang D.Visual object tracking:a survey[J].Pattern Recognition and Artificial Intelligence,2018,31(1):61-76.[卢湖川,李佩霞,王栋.目标跟踪算法综述[J].模式识别与人工智能,2018,31(1):61-76.][DOI:10.16451/j.cnki.issn1003-6059.201801006]
[21]Bertinetto L,Valmadre J,Golodetz S,et al.Staple:complementary learners for real-time tracking[C]//Proceedings of 2016IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,NV,USA:IEEE,2016:1401-1409.[DOI:10.1109/CVPR.2016.156]
[22]Li F,Tian C,Zuo W M,et al.Learning spatial-temporal regularized correlation filters for visual tracking[C]//Proceedings of2018 IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City,UT,USA:IEEE,2018.
[23]Mueller M,Smith N,Ghanem B.Context-aware correlation filter tracking[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,HI,USA:IEEE,2017:1387-1395.[DOI:10.1109/CVPR.2017.152]
[24]Bibi A,Mueller M,Ghanem B.Target response adaptation for correlation filter tracking[C]//Proceedings of 14th European Conference on Computer Vision.Amsterdam,Netherlands:Springer,2016:419-433.[DOI:10.1007/978-3-319-46466-4_25]
[2]Henriques J F,Caseiro R,Martins P,et al.Exploiting the circulant structure of tracking-by-detection with kernels[C]//Proceedings of the 12th European Conference on Computer Vision.Florence,Italy:Springer,2012:702-715.[DOI:10.1007/978-3-642-33765-9 50]
[3]Danelljan M,Khan F S,Felsberg M,et al.Adaptive color attributes for real-time visual tracking[C]//Proceedings of 2014IEEE Conference on Computer Vision and Pattern Recognition.Columbus,OH,USA:IEEE,2014:1090-1097.[DOI:10.1109/CVPR.2014.143]
[4]Henriques J F,Caseiro R,Martins P,et al.High-speed tracking with kernelized correlation filters[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(3):583-596.[DOI:10.1109/TPAMI.2014.2345390]
[5]Tang M,Feng J Y.Multi-kernel correlation filter for visual tracking[C]//Proceedings of 2015 IEEE International Conference on Computer Vision.Santiago,Chile:IEEE,2015:3038-3046.[DOI:10.1109/ICCV.2015.348]
[6]Li Y,Zhu J K.A scale adaptive kernel correlation filter tracker with feature integration[C]//Proceedings of European Conference on Computer Vision.Zurich,Switzerland:Springer,2015:254-265.[DOI:10.1007/978-3-319-16181-5_18]
[7]Danelljan M,Hger G,Khan F S,et al.Accurate scale estimation for robust visual tracking[C]//Proceedings of British Machine Vision Conference.Nottingham,UK:BMVA Press,2014:65.1-65.11.[DOI:10.5244/C.28.65]
[8]Danelljan M,Hger G,Khan F S,et al.Discriminative scale space tracking.[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(8):1561-1575.[DOI:10.1109/TPAMI.2016.2609928]
[9]Galoogahi H K,Sim T,Lucey S.Correlation filters with limited boundaries[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston,MA,USA:IEEE,2015:4630-4638.[DOI:10.1109/CVPR.2015.7299094]
[10]Danelljan M,Hger G,Khan F S,et al.Learning spatially regularized correlation filters for visual tracking[C]//Proceedings of2015 IEEE International Conference on Computer Vision.Santiago,Chile:IEEE,2015:4310-4318.[DOI:10.1109/ICCV.2015.490]
[11]Danelljan M,Robinson A,Khan F S,et al.Beyond correlation filters:learning continuous convolution operators for visual tracking[C]//Proceedings of the 14th European Conference on Computer Vision.Amsterdam,Netherlands:Springer,2016:472-488.[DOI:10.1007/978-3-319-46454-1_29]
[12]Danelljan M,Bhat G,Khan F S,et al.ECO:efficient convolution operators for tracking[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,HI,USA:IEEE,2017:6931-6939.[DOI:10.1109/CVPR.2017.733]
[13]Nam H,Han B.Learning multi-domain convolutional neural networks for visual tracking[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,NV,USA:IEEE,2016:4293-4302.[DOI:10.1109/CVPR.2016.465]
[14]Song Y B,Ma C,Gong L J,et al.CREST:convolutional residual learning for visual tracking[C]//Proceedings of 2017 IEEEInternational Conference on Computer Vision.Venice,Italy:IEEE,2017:2574-2583.[DOI:10.1109/ICCV.2017.279]
[15]Gundogdu E,Alatan A A.Good features to correlate for visual tracking[J].IEEE Transactions on Image Processing,2018,27(5):2526-2540.[DOI:10.1109/TIP.2018.2806280]
[16]Danelljan M,Hger G,Khan F S,et al.Convolutional features for correlation filter based visual tracking[C]//Proceedings of2015 IEEE International Conference on Computer Vision Workshop.Santiago,Chile:IEEE,2015:621-629.[DOI:10.1109/ICCVW.2015.84]
[17]Galoogahi H K,Fagg A,Lucey S.Learning background-aware correlation filters for visual tracking[C]//Proceedings of 2017IEEE International Conference on Computer Vision.Venice,Italy:IEEE,2017:1144-1152.[DOI:10.1109/ICCV.2017.129]
[18]Wu Y,Lim J,Yang M H.Object tracking benchmark[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(9):1834-1848.[DOI:10.1109/TPAMI.2014.2388226]
[19]Zhang W,Kang B S.Recent advances in correlation filter-based object tracking:a review[J].Journal of Image and Graphics,2017,22(8):1017-1033.[张微,康宝生.相关滤波目标跟踪进展综述[J].中国图象图形学报,2017,22(8):1017-1033.][DOI:10.11834/jig.170092]
[20]Lu H C,Li P X,Wang D.Visual object tracking:a survey[J].Pattern Recognition and Artificial Intelligence,2018,31(1):61-76.[卢湖川,李佩霞,王栋.目标跟踪算法综述[J].模式识别与人工智能,2018,31(1):61-76.][DOI:10.16451/j.cnki.issn1003-6059.201801006]
[21]Bertinetto L,Valmadre J,Golodetz S,et al.Staple:complementary learners for real-time tracking[C]//Proceedings of 2016IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,NV,USA:IEEE,2016:1401-1409.[DOI:10.1109/CVPR.2016.156]
[22]Li F,Tian C,Zuo W M,et al.Learning spatial-temporal regularized correlation filters for visual tracking[C]//Proceedings of2018 IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City,UT,USA:IEEE,2018.
[23]Mueller M,Smith N,Ghanem B.Context-aware correlation filter tracking[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,HI,USA:IEEE,2017:1387-1395.[DOI:10.1109/CVPR.2017.152]
[24]Bibi A,Mueller M,Ghanem B.Target response adaptation for correlation filter tracking[C]//Proceedings of 14th European Conference on Computer Vision.Amsterdam,Netherlands:Springer,2016:419-433.[DOI:10.1007/978-3-319-46466-4_25]