足球比赛视频中的目标检测与跟踪
|
摘要
体育节目因为其特有的魅力，在现今的社会中广受大众的欢迎。因此，对体育比赛视频数据的分析研究，具有很高的实际意义和商业价值。本文以足球比赛视频为例，研究了比赛视频中感兴趣的目标——球员、足球和球场标志线的分割提取和预测跟踪。实际应用证明，这三类目标所包含的比赛的信息量最大。
目标分割提取时，本文针对目标的颜色、形状特点，分别采用不同的方法对球场线、球员和足球进行分割检测。对在球场线的分割检测方面，本文对比分析了顶帽变换和边缘检测算子在球场线分割上的效果。由于Hough变换具有对直线残缺和噪声的不敏感性，本文采用该变换方法获取球场线的坐标，并针对Hough变换不能得到直线端点的问题，讨论了一种直线端点的获取方法。同时本文将球场线的坐标在灰度图上进行拟合，以获得更为精确的球场线坐标参数。在球员和足球目标的分割提取方面，本文实现了一系列基于球员和足球的图像分割、候选区域提取以及识别分类的算法。针对图像分割时出现的球员区域中间断裂问题和球员球场线相互粘连问题，本文分析并提出了相应的解决方案。经过大量的实验证明，本文中的目标分割提取算法对噪声具有一定的鲁棒性，在不同的场地情况下，均能获得较为满意的效果。
在目标的预测跟踪时，对于球员和足球的跟踪，本文首先采用Kalman滤波对目标的运动轨迹进行预测估计，结合设置自适应波门来检测目标，实现对球员和足球的跟踪。针对在跟踪中出现的球员匹配和球员间相互遮挡的问题，本文研究并提出了可行的解决方案。对于球场线的预测跟踪，本文提出通过球场线的坐标参数来对球场线的运动趋势进行估计，在球场线的预测区域内进行直线提取操作。同时根据预测的坐标参数，限定Hough变换中两个参数的取值范围，以减小计算量。实验证明，利用本文的跟踪方法可以有效的对足球比赛视频中的目标进行跟踪，使得目标检测的计算量大大减少，其精度也有显著的提高。
本文提出的以足球比赛视频为例的体育视频分析处理技术，已经在实际工程中的虚拟广告生成和虚拟摄像机特效两个方面得到了应用。
Nowadays, the programs of sports video are becoming more and more popular. Therefore, it possesses high actual meaning and commerce worthiness to analyse on the sports video. In this paper, soccer video is analyzed as an example. Detection and tracking of objects in the soccer video are studied. Here the objects we pay attention to include soccer players, ball and court lines. It is proved actually that these three kinds of objects contain a lot of information of the game.
According to the difference in color and sharp, two different measures are adopted to detect the objects we focus on in this paper. About the detection for court lines, this paper discusses and compares the segment effects produced by the Top-Hat transform and the edge detectors. Insensitiveness to the noise, Hough transform is used to obtain coordinate parameters of lines. A method is discussed to get start and end points of lines. About the detection for players and ball, this paper advances a series of algorithms to segment, distill and recognize the areas of players and ball. The problems in segmented images that a clearance often exists in a player area and the conglutination between players and lines are discussed and an appropriate method is advanced to resolve it. Experimental results show that the detection algorithms advanced in this paper is robust to the noise. The results of objects detection are gratifying for various soccer video.
This paper present an algorithm of tracking the objects in the soccer video based on Kalman filter. The tracking history is encoded into state parameter of the filter. The estimated state parameter will be used to reduce search area for model matching. The problems appearing in the tracking process about players matching and players sheltering are studied and methods to resolve them are present. This paper advances that tracking the court lines by estimating coordinate parameters of lines. And the parameters in Hough transform are restricted based on the estimated coordinate parameters to reduce the computation. Experimental results show that the tracking strategy presented in this paper is effective to track objects in soccer video.
The process technologies for the sports video in this paper are already applied in actual projects.
目标分割提取时，本文针对目标的颜色、形状特点，分别采用不同的方法对球场线、球员和足球进行分割检测。对在球场线的分割检测方面，本文对比分析了顶帽变换和边缘检测算子在球场线分割上的效果。由于Hough变换具有对直线残缺和噪声的不敏感性，本文采用该变换方法获取球场线的坐标，并针对Hough变换不能得到直线端点的问题，讨论了一种直线端点的获取方法。同时本文将球场线的坐标在灰度图上进行拟合，以获得更为精确的球场线坐标参数。在球员和足球目标的分割提取方面，本文实现了一系列基于球员和足球的图像分割、候选区域提取以及识别分类的算法。针对图像分割时出现的球员区域中间断裂问题和球员球场线相互粘连问题，本文分析并提出了相应的解决方案。经过大量的实验证明，本文中的目标分割提取算法对噪声具有一定的鲁棒性，在不同的场地情况下，均能获得较为满意的效果。
在目标的预测跟踪时，对于球员和足球的跟踪，本文首先采用Kalman滤波对目标的运动轨迹进行预测估计，结合设置自适应波门来检测目标，实现对球员和足球的跟踪。针对在跟踪中出现的球员匹配和球员间相互遮挡的问题，本文研究并提出了可行的解决方案。对于球场线的预测跟踪，本文提出通过球场线的坐标参数来对球场线的运动趋势进行估计，在球场线的预测区域内进行直线提取操作。同时根据预测的坐标参数，限定Hough变换中两个参数的取值范围，以减小计算量。实验证明，利用本文的跟踪方法可以有效的对足球比赛视频中的目标进行跟踪，使得目标检测的计算量大大减少，其精度也有显著的提高。
本文提出的以足球比赛视频为例的体育视频分析处理技术，已经在实际工程中的虚拟广告生成和虚拟摄像机特效两个方面得到了应用。
Nowadays, the programs of sports video are becoming more and more popular. Therefore, it possesses high actual meaning and commerce worthiness to analyse on the sports video. In this paper, soccer video is analyzed as an example. Detection and tracking of objects in the soccer video are studied. Here the objects we pay attention to include soccer players, ball and court lines. It is proved actually that these three kinds of objects contain a lot of information of the game.
According to the difference in color and sharp, two different measures are adopted to detect the objects we focus on in this paper. About the detection for court lines, this paper discusses and compares the segment effects produced by the Top-Hat transform and the edge detectors. Insensitiveness to the noise, Hough transform is used to obtain coordinate parameters of lines. A method is discussed to get start and end points of lines. About the detection for players and ball, this paper advances a series of algorithms to segment, distill and recognize the areas of players and ball. The problems in segmented images that a clearance often exists in a player area and the conglutination between players and lines are discussed and an appropriate method is advanced to resolve it. Experimental results show that the detection algorithms advanced in this paper is robust to the noise. The results of objects detection are gratifying for various soccer video.
This paper present an algorithm of tracking the objects in the soccer video based on Kalman filter. The tracking history is encoded into state parameter of the filter. The estimated state parameter will be used to reduce search area for model matching. The problems appearing in the tracking process about players matching and players sheltering are studied and methods to resolve them are present. This paper advances that tracking the court lines by estimating coordinate parameters of lines. And the parameters in Hough transform are restricted based on the estimated coordinate parameters to reduce the computation. Experimental results show that the tracking strategy presented in this paper is effective to track objects in soccer video.
The process technologies for the sports video in this paper are already applied in actual projects.
引文
1．高文，陈熙霖．计算机视觉．北京：清华大学出版社，1999
2．刘勋．篮球对象分割和跟踪关键技术研究．硕士学位论文．北京工业大学，2005
3. M.S. Lew, N. Sebe, J.P. Eakins. Content Based Analysis for Video from Snooker Broadcasts. Image and Video Retrieval: International Conference, CIVR 2002,London, UK, July 18-19, 2002. Proceedings: 198-205
4. A. Ekin, A.M. Tekalp. Shot type classification by dominant color for sports video segmentation and summarization. Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03). 2003 IEEE International Conference on, 6-10 April 2003 vol.3: 173-176
5. Bebie T, Bieri H. SoccerMan——Reconstructing Soccer Games from Video Sequences. Proc IEEE International Conference on Image Processing, 1998: 898-902
6. Chen Shu-Ching, Shyu Mei-Ling, Zhang Cheng-cui. Detection of Soccer Goal Shots Using Joint Multimedia Features and Classification Rules. MDM/KDD03, Washington DC, USA. August 27, 2003
7. Y. Rui, A. Gupta, A. Acero. Automatically Extracting Highlights for TV Baseball Programs. In Proc. of ACM Multimedia, 2000: 105-115
8. D.Q. Zhang, R. Raj, S. F. Chang. General and Domain-specific Techniques for Detecting and Recognizing Superimposed Text in Video. In Proc. of IEEE International Conference on Image Processing, 2002
9．Kenneth．R．Castleman．数字图像处理．电子工业出版社，1998
10．赵峙江，张田文，张志宏．一种基于视觉模型与连通域统计的阈值分割新算法．电子学报．2005(5)：793-797
11．朱虹等．数字图像处理基础．北京：科学出版社，2005
12．关向荣．数字视频序列运动目标检测与跟踪技术研究．硕士学位论文．西安：西北工业大学，2005
13．张江山，朱光喜．一种基于Kalman滤波的视频对象跟踪方法．中国图象图形学报．2002(6)：606—609
14．魏娟丽．基于视频序列的人体运动分析系统的研究与实现．硕士学位论文．西安：西北大学，2004
15．冯伟．视频序列运动目标监测与识别方法研究．硕士学位论文．西安：西北工 业大学，2003
16．胡明昊．复杂场景下运动目标的实时跟踪．硕士学位论文．南京：南京理工大学，2002
17．程刚．视频环境中运动目标的检测与跟踪问题研究．硕士学位论文．西安：西安理工大学，2005
18. Y.H.Gong, H.C.Chua, T.S.Lim. Automatic Parsing of TV Soccer Programs. IEEE Trans. on Multimedia, 1995: 167-174
19．刘国翌等．足球场地标志线的自动提取．计算机辅助设计与图形学学报，2003，第15卷第7期：870—874
20. Ahmet.Ekin, A.Murat Tekaip. Automatic Soccer Video Analysis and Summarization. Proc. of SPIE 2003(5021):339-350
21. Y.Ohno, J.Miura, Y.Shirai. Tracking Players and Estimation of the 3D Position of a Ball in Soccer Games. Proc. of ICPR. 2000:145-148
22. An Effective and Fast Soccer Ball Detection and Tracking Method. Proc. of ICPR. 2004
23. Taki T, Hasegawa J, Fukumura T. Development of Motion Analysis System for Quantitative Evaluation of Teamwork in Soccer Games. Proc of IEEE International Conference on Image Processing. 1996, vol Ⅲ:815-818
24．刘彦宏，杜威，李华．足球视频序列中球员的分割与跟踪算法．系统仿真学报，2001(13)：90—93
25．夏德深，傅德胜．现代图像处理技术与应用．南京：东南大学出版社，1997
26．陈震，高满屯，杨声云．基于Hough变换的直线跟踪方法．计算机应用，2003(10)：30—32
27．任金昌，赵荣椿等．一种基于跟踪的全自动直线判别算法．西北工业大学学报，2001年7月，第19卷第3期：427—430
28．张庆利，张兆扬．一种基于运动背景的视频对象分割算法．上海大学学报(自然科学版)，2005年6月，第11卷第2期：112—115
29．刘李杰，蔡德钧，翁南钐．一种面向运动的视频对象分割算法．计算机学报，2000年12月，第23卷第12期：1326—1329
30．张嫣，姚耀文，唐华松．运动图像序列中多目标跟踪的研究与实现．计算机应用研究，2002：74—77
31．张根耀，李竹林，赵宗涛．一种飞行目标跟踪方法．广西大学学报(自然科学版)，2003年9月，第28卷第3期：229—231
32．王艳萍．实时视频图像相关跟踪的算法的改进与实现．舰船科学技术，2004年 6月，第26卷第3期：57—60
33. S. Intille, A. Bobick. Recognizing planned, multi-person action. Comput. Vis.Image Understand. vol. 81, no. 3, Mar. 2001: 414-445
34. K.A. Peker, R. Cabasson, and A. Divakaran. Rapid generation of sports video highlights using the MPEG-7 motion activity descriptor. Proc.SPIE, vol. 4676, Jan.2002: 318-323
35. J. Assfalg, M. Bertini, A. Del Bimbo, W. Nunziati, P. Pala. Soccer highlights detection and recognition using HMMs. in Proc, IEEE Int. Conf. on Mult. And Expo (ICME), Aug. 2002
36．李智勇等．动态图像分析．北京：国防工业出版社，1999
37．贾云得．机器视觉．北京：北京科学出版社，2000
38．何斌，马天予，王运坚，朱红莲．Visual C++数字图像处理．北京：人民邮电出版社．2001
39．张曜，郭立山，李憿玄．Visual C++(MFC)函数实用手册．北京：冶金工业出版社，2004
40．孙即祥．图像处理．北京：科学出版社，2004
41．郑南宁．计算机视觉与模式识别．北京：国防工业出版社，1998
42．孙家广，杨长贵．计算机图形学．北京：清华大学出版社，1994
43. Tomoki Watanabe, Miki Haseyama, Hideo Kitajima. A Soccer Field Tracking Method With Wire Frame Model From TV Images. Proc of IEEE International Conference on Image Processing, 2004:1633-1635
44. Wu Si, Zhang Yong-dong, Lin Shou-xun. An Automatic Segmentation Algorithm for Moving Objects in Video Sequences under Multi-Constraints. IEEE International Conference on Multimedia and Expo, 2004:555-557
45. N. VandenBroucke, L. Macaire, J. G.. Postaire. Soccer players recognition by pixels classification in an hybrid color space. Proc.SPIE, vol. 3071: 23-26
46. Yoshinori Ohno, Jun Miura, Yoshiaki Shirai. Tracking Players and a Ball in Soccer Games. Proceeding of the 1999 IEEE International Conterence on Multisensor Fusion and Integration for Intelligent Systems, August 1999:147-152
47. Pascual Figueroa, Neucimar Leite, Ricardo M. L. Barros. Tracking soccer players using the graph representation. Proceedings of the 17th International Conference on Pattern Recognition, 2004
48. C. Stauffer, W. E. Grimson. Adaptive background mixture for real-time tracking. In Proc. Int. Conf. on Computer Vision and Pattern Recognition, 1999: 246-252
49．陶亮，庄镇泉．复杂背景下人眼自动定位．计算机辅助设计与图形学学报，2003年1月，第15卷第1期：38—41
50. Yu-Lin Kang, Joo-Hwee Lim, Qi Tian, Mohan S. Kankarthalli, Chang-Sheng Xu. Visual Keywords Labeling in Soccer Video. Proceedings of the 17th International Conference on Pattern Recognition,2004
51. P. Xu. Algorithms and Systems for Segmentation and Structure Analysis in Soccer Video. In Proc. of ICME, 2001
2．刘勋．篮球对象分割和跟踪关键技术研究．硕士学位论文．北京工业大学，2005
3. M.S. Lew, N. Sebe, J.P. Eakins. Content Based Analysis for Video from Snooker Broadcasts. Image and Video Retrieval: International Conference, CIVR 2002,London, UK, July 18-19, 2002. Proceedings: 198-205
4. A. Ekin, A.M. Tekalp. Shot type classification by dominant color for sports video segmentation and summarization. Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03). 2003 IEEE International Conference on, 6-10 April 2003 vol.3: 173-176
5. Bebie T, Bieri H. SoccerMan——Reconstructing Soccer Games from Video Sequences. Proc IEEE International Conference on Image Processing, 1998: 898-902
6. Chen Shu-Ching, Shyu Mei-Ling, Zhang Cheng-cui. Detection of Soccer Goal Shots Using Joint Multimedia Features and Classification Rules. MDM/KDD03, Washington DC, USA. August 27, 2003
7. Y. Rui, A. Gupta, A. Acero. Automatically Extracting Highlights for TV Baseball Programs. In Proc. of ACM Multimedia, 2000: 105-115
8. D.Q. Zhang, R. Raj, S. F. Chang. General and Domain-specific Techniques for Detecting and Recognizing Superimposed Text in Video. In Proc. of IEEE International Conference on Image Processing, 2002
9．Kenneth．R．Castleman．数字图像处理．电子工业出版社，1998
10．赵峙江，张田文，张志宏．一种基于视觉模型与连通域统计的阈值分割新算法．电子学报．2005(5)：793-797
11．朱虹等．数字图像处理基础．北京：科学出版社，2005
12．关向荣．数字视频序列运动目标检测与跟踪技术研究．硕士学位论文．西安：西北工业大学，2005
13．张江山，朱光喜．一种基于Kalman滤波的视频对象跟踪方法．中国图象图形学报．2002(6)：606—609
14．魏娟丽．基于视频序列的人体运动分析系统的研究与实现．硕士学位论文．西安：西北大学，2004
15．冯伟．视频序列运动目标监测与识别方法研究．硕士学位论文．西安：西北工 业大学，2003
16．胡明昊．复杂场景下运动目标的实时跟踪．硕士学位论文．南京：南京理工大学，2002
17．程刚．视频环境中运动目标的检测与跟踪问题研究．硕士学位论文．西安：西安理工大学，2005
18. Y.H.Gong, H.C.Chua, T.S.Lim. Automatic Parsing of TV Soccer Programs. IEEE Trans. on Multimedia, 1995: 167-174
19．刘国翌等．足球场地标志线的自动提取．计算机辅助设计与图形学学报，2003，第15卷第7期：870—874
20. Ahmet.Ekin, A.Murat Tekaip. Automatic Soccer Video Analysis and Summarization. Proc. of SPIE 2003(5021):339-350
21. Y.Ohno, J.Miura, Y.Shirai. Tracking Players and Estimation of the 3D Position of a Ball in Soccer Games. Proc. of ICPR. 2000:145-148
22. An Effective and Fast Soccer Ball Detection and Tracking Method. Proc. of ICPR. 2004
23. Taki T, Hasegawa J, Fukumura T. Development of Motion Analysis System for Quantitative Evaluation of Teamwork in Soccer Games. Proc of IEEE International Conference on Image Processing. 1996, vol Ⅲ:815-818
24．刘彦宏，杜威，李华．足球视频序列中球员的分割与跟踪算法．系统仿真学报，2001(13)：90—93
25．夏德深，傅德胜．现代图像处理技术与应用．南京：东南大学出版社，1997
26．陈震，高满屯，杨声云．基于Hough变换的直线跟踪方法．计算机应用，2003(10)：30—32
27．任金昌，赵荣椿等．一种基于跟踪的全自动直线判别算法．西北工业大学学报，2001年7月，第19卷第3期：427—430
28．张庆利，张兆扬．一种基于运动背景的视频对象分割算法．上海大学学报(自然科学版)，2005年6月，第11卷第2期：112—115
29．刘李杰，蔡德钧，翁南钐．一种面向运动的视频对象分割算法．计算机学报，2000年12月，第23卷第12期：1326—1329
30．张嫣，姚耀文，唐华松．运动图像序列中多目标跟踪的研究与实现．计算机应用研究，2002：74—77
31．张根耀，李竹林，赵宗涛．一种飞行目标跟踪方法．广西大学学报(自然科学版)，2003年9月，第28卷第3期：229—231
32．王艳萍．实时视频图像相关跟踪的算法的改进与实现．舰船科学技术，2004年 6月，第26卷第3期：57—60
33. S. Intille, A. Bobick. Recognizing planned, multi-person action. Comput. Vis.Image Understand. vol. 81, no. 3, Mar. 2001: 414-445
34. K.A. Peker, R. Cabasson, and A. Divakaran. Rapid generation of sports video highlights using the MPEG-7 motion activity descriptor. Proc.SPIE, vol. 4676, Jan.2002: 318-323
35. J. Assfalg, M. Bertini, A. Del Bimbo, W. Nunziati, P. Pala. Soccer highlights detection and recognition using HMMs. in Proc, IEEE Int. Conf. on Mult. And Expo (ICME), Aug. 2002
36．李智勇等．动态图像分析．北京：国防工业出版社，1999
37．贾云得．机器视觉．北京：北京科学出版社，2000
38．何斌，马天予，王运坚，朱红莲．Visual C++数字图像处理．北京：人民邮电出版社．2001
39．张曜，郭立山，李憿玄．Visual C++(MFC)函数实用手册．北京：冶金工业出版社，2004
40．孙即祥．图像处理．北京：科学出版社，2004
41．郑南宁．计算机视觉与模式识别．北京：国防工业出版社，1998
42．孙家广，杨长贵．计算机图形学．北京：清华大学出版社，1994
43. Tomoki Watanabe, Miki Haseyama, Hideo Kitajima. A Soccer Field Tracking Method With Wire Frame Model From TV Images. Proc of IEEE International Conference on Image Processing, 2004:1633-1635
44. Wu Si, Zhang Yong-dong, Lin Shou-xun. An Automatic Segmentation Algorithm for Moving Objects in Video Sequences under Multi-Constraints. IEEE International Conference on Multimedia and Expo, 2004:555-557
45. N. VandenBroucke, L. Macaire, J. G.. Postaire. Soccer players recognition by pixels classification in an hybrid color space. Proc.SPIE, vol. 3071: 23-26
46. Yoshinori Ohno, Jun Miura, Yoshiaki Shirai. Tracking Players and a Ball in Soccer Games. Proceeding of the 1999 IEEE International Conterence on Multisensor Fusion and Integration for Intelligent Systems, August 1999:147-152
47. Pascual Figueroa, Neucimar Leite, Ricardo M. L. Barros. Tracking soccer players using the graph representation. Proceedings of the 17th International Conference on Pattern Recognition, 2004
48. C. Stauffer, W. E. Grimson. Adaptive background mixture for real-time tracking. In Proc. Int. Conf. on Computer Vision and Pattern Recognition, 1999: 246-252
49．陶亮，庄镇泉．复杂背景下人眼自动定位．计算机辅助设计与图形学学报，2003年1月，第15卷第1期：38—41
50. Yu-Lin Kang, Joo-Hwee Lim, Qi Tian, Mohan S. Kankarthalli, Chang-Sheng Xu. Visual Keywords Labeling in Soccer Video. Proceedings of the 17th International Conference on Pattern Recognition,2004
51. P. Xu. Algorithms and Systems for Segmentation and Structure Analysis in Soccer Video. In Proc. of ICME, 2001