基于半监督在线学习的跟踪算法研究
|
摘要
目标跟踪,属于计算机视觉的研究范畴,作为计算机视觉领域中一个极具吸引力又极富挑战性的课题,一直得到研究工作者们的广泛关注和投入。利用机器学习方法解决跟踪问题是最近兴起的热点研究领域,也代表了未来目标跟踪领域的方向。本文对基于限制条件的半监督学习算法在目标跟踪中的应用进行了研究,主要工作如下:
1.在分析与对比现有的半监督学习算法的基础上,对P-N Learning进行了深入研究,并通过实验验证了P-N Learning的性能。
2.对多种跟踪算法进行了研究。主要包括经典的光流法,新近流行的仿射粒子滤波算法以广义霍夫变换在目标跟踪中的应用。将光流法和仿射粒子滤波分别纳入本文设计的半监督学习跟踪系统框架。在广义霍夫变换的研究方面,实现了一种基于广义霍夫变换的融合局部特征匹配与概率外观模型的多目标跟踪方法。
3.设计实现了一种基于半监督在线学习的跟踪系统框架。利用多层限制条件对训练样本选取和分类器训练进行干预,提升了分类器性能。并分别将光流法和仿射粒子滤波纳入系统框架,实现了完整的跟踪功能。实验证明,本文系统在目标快速运动、目标旋转、目标消失重现、外观相似目标遮挡等情况下,都能取得较为理想的跟踪效果。
Object tracking is an attractive and challenging subject of computer vision, draws a lot of attention. Solving tracking problem with machine learning technique is a new rising area of research, also indicates the trend of tracking research in the future. In this thesis, we mainly study the application of semi-supervised learning in object tracking area, especially the one based on structural constraints. It’s organized as follows:
1. Semi-supervised learning algorithm Several popular semi-supervised learning algorithms are studied, including semi-boosting, multiple instance learning, and P-N learning. Main attention is paid to P-N learning, especially its performance in object detection and tracking area.
2. Research on tracking algorithm We investigate several popular tracking algorithms, including optical flow, particle filter on affine group, and generalized hough transform. Optical flow and particle filter on affine group are incorporated in our semi-supervised learning based tracking framework. Moreover, A new approach for tracking multiple objects is proposed, which combines feature correspondence with a probabilistic appearance model, and uses generalized hough transform to determine the optimal target position.
3. Design and implementation of tracking system based on semi-supervised learning
Temporal, spatial and data correlation constraints are designed to intervene extraction of training data, training of classifiers and labeling of unlabeled data, which improves the performance of classifiers. Optical flow and particle filter on affine group are incorporated in our tracking system framework. Experiments show that our tracking system work well under challenging situation, including fast moving, large scale rotation, object reappear and occlusion with similar object.
1.在分析与对比现有的半监督学习算法的基础上,对P-N Learning进行了深入研究,并通过实验验证了P-N Learning的性能。
2.对多种跟踪算法进行了研究。主要包括经典的光流法,新近流行的仿射粒子滤波算法以广义霍夫变换在目标跟踪中的应用。将光流法和仿射粒子滤波分别纳入本文设计的半监督学习跟踪系统框架。在广义霍夫变换的研究方面,实现了一种基于广义霍夫变换的融合局部特征匹配与概率外观模型的多目标跟踪方法。
3.设计实现了一种基于半监督在线学习的跟踪系统框架。利用多层限制条件对训练样本选取和分类器训练进行干预,提升了分类器性能。并分别将光流法和仿射粒子滤波纳入系统框架,实现了完整的跟踪功能。实验证明,本文系统在目标快速运动、目标旋转、目标消失重现、外观相似目标遮挡等情况下,都能取得较为理想的跟踪效果。
Object tracking is an attractive and challenging subject of computer vision, draws a lot of attention. Solving tracking problem with machine learning technique is a new rising area of research, also indicates the trend of tracking research in the future. In this thesis, we mainly study the application of semi-supervised learning in object tracking area, especially the one based on structural constraints. It’s organized as follows:
1. Semi-supervised learning algorithm Several popular semi-supervised learning algorithms are studied, including semi-boosting, multiple instance learning, and P-N learning. Main attention is paid to P-N learning, especially its performance in object detection and tracking area.
2. Research on tracking algorithm We investigate several popular tracking algorithms, including optical flow, particle filter on affine group, and generalized hough transform. Optical flow and particle filter on affine group are incorporated in our semi-supervised learning based tracking framework. Moreover, A new approach for tracking multiple objects is proposed, which combines feature correspondence with a probabilistic appearance model, and uses generalized hough transform to determine the optimal target position.
3. Design and implementation of tracking system based on semi-supervised learning
Temporal, spatial and data correlation constraints are designed to intervene extraction of training data, training of classifiers and labeling of unlabeled data, which improves the performance of classifiers. Optical flow and particle filter on affine group are incorporated in our tracking system framework. Experiments show that our tracking system work well under challenging situation, including fast moving, large scale rotation, object reappear and occlusion with similar object.
引文
[1] D.A.Forsyth and J.Ponce.Computer vision:a modern approach.Prentice Hall Press,2008.8
[2] Collins R.T., Lipton A.J., Kanade T., A system for video surveillance and monitoring, Proceedings of the American Nuclear Society (ANS) 8th International Topical Meeting on Robotic and Remote Systems, 1999.
[3] Haritaoglu I., Harwood D., Davis L.S., W4: Real-time surveillance of people and their activities, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22 (8), pp. 809-830.
[4] Grimson W.E.L., Stauffer C., Romano R., Lee L., Using adaptive tracking to classify and monitor activities in a site, In Proceedings of IEEE Computer Vision and Pattern Recognition, 1998, pp. 22-29.
[5] M. J. Black and A. D. Jepson. EigenTracking: Robust matching and tracking of articulated objects using a view-based representation. International Journal of Computer Vision, 1998, 26, 63-84.
[6]程建,周越,蔡念,杨杰.基于粒子滤波的红外目标跟踪.红外与毫米波学报, 2006, 25, 113-117.
[7]刘隆和编著,多模复合寻的制导技术,北京:国防工业出版社, 1998.
[8] A. Yilmaz, K. Shafique, M. Shah. Target tracking in airborne forward looking infrared imagery.Image and Vision Computing. 2003, 21, 623-635.
[9] N. P. Papanikolopoulos, P. K. Khosla, T. Kanade. Visual tracking of a moving target by a camera mounted on a robot: a combination of control and vision. IEEE Transactions on Robotics and Automation, 1993, 9, 14-35.
[10] B. Jung, G. S. Sukhatme. A generalized region-based approach for multi-target tracking in outdoor environments. Proceedings of IEEE International Conference on Robotics and Automation, 2004, 2189-2195.
[11] B. Jung, G. S. Sukhatme. Detecting moving objects using a single camera on a mobile robot in an outdoor environment. Proceedings of International Conference on Intelligent Autonomous Systems, 2004, 980–987.
[12] H. Nait-Charif, S. J. McKenna. Head tracking and action recognition in a smart meeting room. Proceedings of IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, 2003, 24-31.
[13] J. MacCormick, A. Blake. A probabilistic exclusion principle for tracking multiple objects. International Journal of Computer Vision, 2000, 39, 57-71.
[14] B. Coifman, D. Beymer, P. Mclauchlan, J. Malik. A real-time computer vision system for vehicle tracking and traffic surveillance. Transportation Research Part C, 6, 271-288.
[15] D. Magee. Tracking multiple vehicles using foreground, background and motion models. Image and Vision Computing, 2004, 22, 143-155.
[16] V.Keanaker and R.Zabih.Bayesian Multi-Camera Surveillance,IEEE Computer Society Conference on Computer Vision and Pattern Recognition,1999,1(2):253-259.
[17]彭甜,多摄像头多目标跟踪技术研究,上海交通大学[学位论文],2010
[18]侯志强,韩崇昭.视觉跟踪技术综述A Survey of Visual Tracking [自动化学报Acta Automatica Sinica], 2006年04期.
[19] D.Comaniciu andP.Meer.Meanshift:A robust approach toward features space analysis. IEEE Transactions on pattern analysis and machine intelligence, Pages 603-619,2002.
[20] D.Comaniciu and V.Ramesh. Real-time traeking of non-rigid objects using mean shift,July 8 2003,US Patent6,590,999.
[21] Andrieu C., de Freitas N., Doucet A., Jordan M.I., An introduction to MCMC for machine learning, Machine Learning, 2003, 50, pp. 5-43.
[22] Liu J.S., Metropolized independent sampling with comparison to rejection sampling and importance sampling, 1996, 6, pp. 133-119.
[23] Robert C.P., Casella G., Monte Carlo statistical methods, New York: Springer-Verlag, 1999.
[24] Geweke J., Bayesian inference in econometrics models using Monte Carlo integration, Econometrica, 1989, 57, pp. 1317-1339.
[25] Doucet, A., de Freitas, J.F.G., Gordon N.J. Sequential Monte Carlo methods in practice. New York: Springer-Verlag, 2001.
[26] D.Ross,J.Lim,R.Lin and M.Yang.Incremental learning for robust visual tracking,IJCV, May 2008,77(1):125–141.
[27] H.Grabner,M.Grabner,and H.Bischof.Real-time tracking via online boosting.In BMVC, 2006,47–56.
[28] Helmut Grabner and Horst Bischof, On-line Boosting and Vision
[29] Boris Babenko,Ming-Hsuan Yang,Serge Belongie,Visual Tracking with Online Multiple Instance Learning
[30] Helmut Grabner Christian Leistner and Horst Bischof, Semi-Supervised On-line Boosting for Robust Tracking
[31] Severin Stalder,Helmut Grabner,Luc van Gool, Beyond Semi-Supervised Tracking:Tracking Should Be as Simple as Detection, but not Simpler than Recognition
[32] Zdenek Kalal,Jiri Matas,Krystian Mikolajczyk, P-NLearning: Bootstrapping Binary Classifiers by Structural Constraints
[33] Y. Freund and R. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):119–139, 1997.
[34] N. Oza and S. Russell. Online bagging and boosting. InProc.Artificial Intelligence and Statistics, pages 105–112, 2001.
[35] Grabner, H., Bischof, H.: On-line boosting and vision. In: Proc. CVPR. Volume 1.(2006) 260-267
[36] P. Viola, J. C. Platt, and C. Zhang. Multiple instance boosting for object detection. InNIPS, pages 1417–1426, 2005.
[37] Beauchemin S S,Barron J L.The computation of optical flow.ACM Computing Surveys,1995,27(3):433~467
[38] Barron J,Fleet D,Beauchemin S S.Performance of Optical Flow Techniques. Intemational Journal of Computer Vision,1994,12(l):43~77
[39] Visual Tracking via Incremental Self-tuning Particle Filtering on the Affine Group
[40] O. Tuzel, F. Porikli, and P. Meer. Learning on lie groups for invariant detection and tracking. Proc. of CVPR’08, 2008.
[41] S. Vijayakumar, A. D. Souza, and S. Schaal. Incremental online learning in high dimensions. Neural Computation,17:2602–2634, 2005.
[42] J. Kwon, K. M. Lee, and F. C. Park. Visual tracking via geometric particle filtering on the affine group with optimal importance functions. Proc. of CVPR’09, 2009.
[43] J.Gall and V.Lempitsky, "Class-Specific Hough Forests for Object Detection," IEEE Conference on Computer Vision and Pattern Recognition(2009).
[44] M.Godec, P.M.Roth, and H.Bischof, "Hough-based Tracking of Non-Rigid Objects," ICCV2011
[45] B. Lei and L.-Q. Xu. Real-time outdoor video surveillance with robust foreground extraction and object tracking via multi-state transition management[J]. Pattern Recognition Letters, 2006,27(15):1816–1825.
[46] A. Senior, A. Hampapur, Y.-L. Tian,et al. Appearance Models for Occlusion Handling[J]. Intl.J. Image and Vision Computing,2006, 24(11):1233-1243 .
[47] A. D. Jepson, D. J. Fleet and T. F. EI-Maraghi. Robust Online Appearance Models for Visual Tracking[J]. IEEE Trans. Pattern Anal. Mach. Intell, 2003,25(10): 1296–1311.
[48] H.T. Nguyen and A.W. M. Smeulders. Fast occluded object tracking by a robust appearance ?lter[J]. IEEE Trans. Pattern Anal. Mach. Intell, 2004,268: 1099–1104
[49] J.Gall and V.Lempitsky.Class-Specific Hough Forests for Object Detection[A].IEEE Conf. on Computer Vision and Pattern Recognition,2009.
[50] B. Leibe, A. Leonardis, and B. Schiele.Robust Object Detection with Interleaved Categorization and Segmentation[J]. Intl.J. Computer Vision,2008,77(1–3): 259–289.
[51] A.BlakeandM.Isard. Active contours: the application of techniques from graphics, vision,control theory and statistics to visual tracking of shapes motion. Springer-Verlag New York Inc.Secaucus, NJ, USA, 1998.
[52] R.Malladi, J.A.Sethian, B.C.Vemuri, et al. Shape modeling with front propagation: A level set approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(2):158-175,1995.
[53] V.Caselles, R.Kimmel, and G.Sapiro. Geodesic active contours. International journal of computer vision, 22(1):61-79, 1997.
[54] N.Paragios and R.Deriehe. Geodesic active contours and level sets for the detection and tracking of moving objects. IEEE Transactions on Pattern Analysis and Machine Intelligence,22(3):266一280, 2000.
[55] S.Kim, C.B.Park, and S.W.Lee. Tracking 3D human body using Partiele filter in moving monocular camera. In Pattern Recognition, 2006. ICPR 2006.18th International Conference on Pattern Recognition,volume4,Pages805-808,2006.
[56] B.Fan and Z.F.Wang. Pose estimation of human body based on silhouette images. In Information Acquisition,2004. Proceedings International Conference on Information Acquisition, Pages296-300,2004.
[57] B.Stenger, PRS Mendonca, and R.Cipolla. Model-based 3D tracking of an articulated hand. image,23:4.
[58] T.X.Han and T.S.Huang. Articulated body tracking using dynamic belief propagation. Lecture notes in computer science,3766:26,2005.
[59] Q.Delamarre and O.Faugeras.3D articulated models and multi-view tracking with silhouettes. In Proc. of ICCV,volume99,Pages716-721,1999.
[60] Breiman, Leo (2001). "Random Forests". Machine Learning 45 (1):5–32.
[61] Ho, Tin (1995). "Random Decision Forest". 3rd Int'l Conf. on Document Analysis and Recognition. pp. 278–282.
[62] Ho, Tin (1998). "The Random Subspace Method for Constructing Decision Forests". IEEE Transactions on Pattern Analysis and Machine Intelligence 20 (8): 832–844.
[2] Collins R.T., Lipton A.J., Kanade T., A system for video surveillance and monitoring, Proceedings of the American Nuclear Society (ANS) 8th International Topical Meeting on Robotic and Remote Systems, 1999.
[3] Haritaoglu I., Harwood D., Davis L.S., W4: Real-time surveillance of people and their activities, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22 (8), pp. 809-830.
[4] Grimson W.E.L., Stauffer C., Romano R., Lee L., Using adaptive tracking to classify and monitor activities in a site, In Proceedings of IEEE Computer Vision and Pattern Recognition, 1998, pp. 22-29.
[5] M. J. Black and A. D. Jepson. EigenTracking: Robust matching and tracking of articulated objects using a view-based representation. International Journal of Computer Vision, 1998, 26, 63-84.
[6]程建,周越,蔡念,杨杰.基于粒子滤波的红外目标跟踪.红外与毫米波学报, 2006, 25, 113-117.
[7]刘隆和编著,多模复合寻的制导技术,北京:国防工业出版社, 1998.
[8] A. Yilmaz, K. Shafique, M. Shah. Target tracking in airborne forward looking infrared imagery.Image and Vision Computing. 2003, 21, 623-635.
[9] N. P. Papanikolopoulos, P. K. Khosla, T. Kanade. Visual tracking of a moving target by a camera mounted on a robot: a combination of control and vision. IEEE Transactions on Robotics and Automation, 1993, 9, 14-35.
[10] B. Jung, G. S. Sukhatme. A generalized region-based approach for multi-target tracking in outdoor environments. Proceedings of IEEE International Conference on Robotics and Automation, 2004, 2189-2195.
[11] B. Jung, G. S. Sukhatme. Detecting moving objects using a single camera on a mobile robot in an outdoor environment. Proceedings of International Conference on Intelligent Autonomous Systems, 2004, 980–987.
[12] H. Nait-Charif, S. J. McKenna. Head tracking and action recognition in a smart meeting room. Proceedings of IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, 2003, 24-31.
[13] J. MacCormick, A. Blake. A probabilistic exclusion principle for tracking multiple objects. International Journal of Computer Vision, 2000, 39, 57-71.
[14] B. Coifman, D. Beymer, P. Mclauchlan, J. Malik. A real-time computer vision system for vehicle tracking and traffic surveillance. Transportation Research Part C, 6, 271-288.
[15] D. Magee. Tracking multiple vehicles using foreground, background and motion models. Image and Vision Computing, 2004, 22, 143-155.
[16] V.Keanaker and R.Zabih.Bayesian Multi-Camera Surveillance,IEEE Computer Society Conference on Computer Vision and Pattern Recognition,1999,1(2):253-259.
[17]彭甜,多摄像头多目标跟踪技术研究,上海交通大学[学位论文],2010
[18]侯志强,韩崇昭.视觉跟踪技术综述A Survey of Visual Tracking [自动化学报Acta Automatica Sinica], 2006年04期.
[19] D.Comaniciu andP.Meer.Meanshift:A robust approach toward features space analysis. IEEE Transactions on pattern analysis and machine intelligence, Pages 603-619,2002.
[20] D.Comaniciu and V.Ramesh. Real-time traeking of non-rigid objects using mean shift,July 8 2003,US Patent6,590,999.
[21] Andrieu C., de Freitas N., Doucet A., Jordan M.I., An introduction to MCMC for machine learning, Machine Learning, 2003, 50, pp. 5-43.
[22] Liu J.S., Metropolized independent sampling with comparison to rejection sampling and importance sampling, 1996, 6, pp. 133-119.
[23] Robert C.P., Casella G., Monte Carlo statistical methods, New York: Springer-Verlag, 1999.
[24] Geweke J., Bayesian inference in econometrics models using Monte Carlo integration, Econometrica, 1989, 57, pp. 1317-1339.
[25] Doucet, A., de Freitas, J.F.G., Gordon N.J. Sequential Monte Carlo methods in practice. New York: Springer-Verlag, 2001.
[26] D.Ross,J.Lim,R.Lin and M.Yang.Incremental learning for robust visual tracking,IJCV, May 2008,77(1):125–141.
[27] H.Grabner,M.Grabner,and H.Bischof.Real-time tracking via online boosting.In BMVC, 2006,47–56.
[28] Helmut Grabner and Horst Bischof, On-line Boosting and Vision
[29] Boris Babenko,Ming-Hsuan Yang,Serge Belongie,Visual Tracking with Online Multiple Instance Learning
[30] Helmut Grabner Christian Leistner and Horst Bischof, Semi-Supervised On-line Boosting for Robust Tracking
[31] Severin Stalder,Helmut Grabner,Luc van Gool, Beyond Semi-Supervised Tracking:Tracking Should Be as Simple as Detection, but not Simpler than Recognition
[32] Zdenek Kalal,Jiri Matas,Krystian Mikolajczyk, P-NLearning: Bootstrapping Binary Classifiers by Structural Constraints
[33] Y. Freund and R. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):119–139, 1997.
[34] N. Oza and S. Russell. Online bagging and boosting. InProc.Artificial Intelligence and Statistics, pages 105–112, 2001.
[35] Grabner, H., Bischof, H.: On-line boosting and vision. In: Proc. CVPR. Volume 1.(2006) 260-267
[36] P. Viola, J. C. Platt, and C. Zhang. Multiple instance boosting for object detection. InNIPS, pages 1417–1426, 2005.
[37] Beauchemin S S,Barron J L.The computation of optical flow.ACM Computing Surveys,1995,27(3):433~467
[38] Barron J,Fleet D,Beauchemin S S.Performance of Optical Flow Techniques. Intemational Journal of Computer Vision,1994,12(l):43~77
[39] Visual Tracking via Incremental Self-tuning Particle Filtering on the Affine Group
[40] O. Tuzel, F. Porikli, and P. Meer. Learning on lie groups for invariant detection and tracking. Proc. of CVPR’08, 2008.
[41] S. Vijayakumar, A. D. Souza, and S. Schaal. Incremental online learning in high dimensions. Neural Computation,17:2602–2634, 2005.
[42] J. Kwon, K. M. Lee, and F. C. Park. Visual tracking via geometric particle filtering on the affine group with optimal importance functions. Proc. of CVPR’09, 2009.
[43] J.Gall and V.Lempitsky, "Class-Specific Hough Forests for Object Detection," IEEE Conference on Computer Vision and Pattern Recognition(2009).
[44] M.Godec, P.M.Roth, and H.Bischof, "Hough-based Tracking of Non-Rigid Objects," ICCV2011
[45] B. Lei and L.-Q. Xu. Real-time outdoor video surveillance with robust foreground extraction and object tracking via multi-state transition management[J]. Pattern Recognition Letters, 2006,27(15):1816–1825.
[46] A. Senior, A. Hampapur, Y.-L. Tian,et al. Appearance Models for Occlusion Handling[J]. Intl.J. Image and Vision Computing,2006, 24(11):1233-1243 .
[47] A. D. Jepson, D. J. Fleet and T. F. EI-Maraghi. Robust Online Appearance Models for Visual Tracking[J]. IEEE Trans. Pattern Anal. Mach. Intell, 2003,25(10): 1296–1311.
[48] H.T. Nguyen and A.W. M. Smeulders. Fast occluded object tracking by a robust appearance ?lter[J]. IEEE Trans. Pattern Anal. Mach. Intell, 2004,268: 1099–1104
[49] J.Gall and V.Lempitsky.Class-Specific Hough Forests for Object Detection[A].IEEE Conf. on Computer Vision and Pattern Recognition,2009.
[50] B. Leibe, A. Leonardis, and B. Schiele.Robust Object Detection with Interleaved Categorization and Segmentation[J]. Intl.J. Computer Vision,2008,77(1–3): 259–289.
[51] A.BlakeandM.Isard. Active contours: the application of techniques from graphics, vision,control theory and statistics to visual tracking of shapes motion. Springer-Verlag New York Inc.Secaucus, NJ, USA, 1998.
[52] R.Malladi, J.A.Sethian, B.C.Vemuri, et al. Shape modeling with front propagation: A level set approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(2):158-175,1995.
[53] V.Caselles, R.Kimmel, and G.Sapiro. Geodesic active contours. International journal of computer vision, 22(1):61-79, 1997.
[54] N.Paragios and R.Deriehe. Geodesic active contours and level sets for the detection and tracking of moving objects. IEEE Transactions on Pattern Analysis and Machine Intelligence,22(3):266一280, 2000.
[55] S.Kim, C.B.Park, and S.W.Lee. Tracking 3D human body using Partiele filter in moving monocular camera. In Pattern Recognition, 2006. ICPR 2006.18th International Conference on Pattern Recognition,volume4,Pages805-808,2006.
[56] B.Fan and Z.F.Wang. Pose estimation of human body based on silhouette images. In Information Acquisition,2004. Proceedings International Conference on Information Acquisition, Pages296-300,2004.
[57] B.Stenger, PRS Mendonca, and R.Cipolla. Model-based 3D tracking of an articulated hand. image,23:4.
[58] T.X.Han and T.S.Huang. Articulated body tracking using dynamic belief propagation. Lecture notes in computer science,3766:26,2005.
[59] Q.Delamarre and O.Faugeras.3D articulated models and multi-view tracking with silhouettes. In Proc. of ICCV,volume99,Pages716-721,1999.
[60] Breiman, Leo (2001). "Random Forests". Machine Learning 45 (1):5–32.
[61] Ho, Tin (1995). "Random Decision Forest". 3rd Int'l Conf. on Document Analysis and Recognition. pp. 278–282.
[62] Ho, Tin (1998). "The Random Subspace Method for Constructing Decision Forests". IEEE Transactions on Pattern Analysis and Machine Intelligence 20 (8): 832–844.