基于视觉的人体跟踪技术在人机交互中的应用
|
摘要
现阶段,数字娱乐领域的人机交互模式单一,此研究将计算机视觉引入数字娱乐的人机交互,采用基于视觉的人体运动分析作为数字娱乐的人机游戏交互界面,进一步丰富和完善了数字娱乐的人机接口。
本研究应用了一个简易的人体上半身三维模型,这个模型对跟踪和人体模型的恢复有很强的指导作用,能够很好地模拟人上半身的一般动作。使用USB单摄像头采集图像,实时计算场景深度信息,然后结合使用深度和颜色信息进行人体跟踪,做了基于颜色直方图的粒子滤波的跟踪算法,并得出了实验结果。通过实验证明,该系统能在复杂背景下对人体上半身的一般运动进行实时鲁棒的跟踪和识别,并很好地解决了自动初始化和错误恢复等问题。此外还开发了一套数字娱乐程序框架和一款运用了DirectX技术,完全使用人体运动分析系统且通过身体驱动的Stranded游戏,运用Body Tracking类作为该游戏的输入系统,该类是输入系统中的最主要的一个类。它是把基于视觉的人体跟踪技术运用在游戏中的人机交互接口的平台,以手部姿态跟踪与识别为例进行人机交互的实现,头部的跟踪与识别则参照上述方法和程序进行实现。
本游戏系统通过从USB摄像机中获取的人体实时图像,经过图像的预处理(图像的简化等),得到不同阙值的人体头部和人手的特征量,并从中选取当前背景下最合适的阙值,让决策树进行识别,最后通过窗口控制来作为游戏中替代鼠标键盘的输入方式,实现游戏的人机交互,从而构建了一个新颖的具有很强人机交互特点的数字娱乐系统。
Now,for the single pattern of human-computer interaction in digital entertainment,in this thesis, computer vision is introduced into human-computer interaction in digital entertainment to perfect the practice of man-machine interface by the analysis on the body movement based on computer visual.
The thesis introduce a simple 3D model of the upper half of the body which can imitate body movement and has an important direction in tracking and recovering of body model.The experiment is performanced that USB camera is used to get image data, to simultaneously calculate the depth information, then to apply a body track by using the information of depth and color and to perform a tracking algorithm of particle filtering based on color histogram. The results of the experiment indicate that the Robust tracking and recognizing of normal movement of the upper body can be realized in complicated background and such problems as automatic initialization and fault recovering can be resolved.Moreover,digital entertainment program framework is built and Stranded Game is developed through Direct X technology which applies the analysis system on the body movement and takes the body as a driver. It is the most important one of all input systems, a computer platform of human-computer Interaction which body tracking is applied based on visual in the Computer Game.It realizes the human-computer function by taking hand movement recognizing and tracking for example which head recognizing and tracking is also carried out referring to the method and program mentioned above.
The system of the game takes real-time body image by USB in the telecamera, gets different fault value of characteristic feature of head and hand and then chooses the most suitabl fault value to be recognized for decision tree and at last realizes human-computer interaction by taking window controll as input system taking the place of mouse and keyboard in the game. .The above all build a new digital entertainment system with the specialty of human-computer Interaction.
本研究应用了一个简易的人体上半身三维模型,这个模型对跟踪和人体模型的恢复有很强的指导作用,能够很好地模拟人上半身的一般动作。使用USB单摄像头采集图像,实时计算场景深度信息,然后结合使用深度和颜色信息进行人体跟踪,做了基于颜色直方图的粒子滤波的跟踪算法,并得出了实验结果。通过实验证明,该系统能在复杂背景下对人体上半身的一般运动进行实时鲁棒的跟踪和识别,并很好地解决了自动初始化和错误恢复等问题。此外还开发了一套数字娱乐程序框架和一款运用了DirectX技术,完全使用人体运动分析系统且通过身体驱动的Stranded游戏,运用Body Tracking类作为该游戏的输入系统,该类是输入系统中的最主要的一个类。它是把基于视觉的人体跟踪技术运用在游戏中的人机交互接口的平台,以手部姿态跟踪与识别为例进行人机交互的实现,头部的跟踪与识别则参照上述方法和程序进行实现。
本游戏系统通过从USB摄像机中获取的人体实时图像,经过图像的预处理(图像的简化等),得到不同阙值的人体头部和人手的特征量,并从中选取当前背景下最合适的阙值,让决策树进行识别,最后通过窗口控制来作为游戏中替代鼠标键盘的输入方式,实现游戏的人机交互,从而构建了一个新颖的具有很强人机交互特点的数字娱乐系统。
Now,for the single pattern of human-computer interaction in digital entertainment,in this thesis, computer vision is introduced into human-computer interaction in digital entertainment to perfect the practice of man-machine interface by the analysis on the body movement based on computer visual.
The thesis introduce a simple 3D model of the upper half of the body which can imitate body movement and has an important direction in tracking and recovering of body model.The experiment is performanced that USB camera is used to get image data, to simultaneously calculate the depth information, then to apply a body track by using the information of depth and color and to perform a tracking algorithm of particle filtering based on color histogram. The results of the experiment indicate that the Robust tracking and recognizing of normal movement of the upper body can be realized in complicated background and such problems as automatic initialization and fault recovering can be resolved.Moreover,digital entertainment program framework is built and Stranded Game is developed through Direct X technology which applies the analysis system on the body movement and takes the body as a driver. It is the most important one of all input systems, a computer platform of human-computer Interaction which body tracking is applied based on visual in the Computer Game.It realizes the human-computer function by taking hand movement recognizing and tracking for example which head recognizing and tracking is also carried out referring to the method and program mentioned above.
The system of the game takes real-time body image by USB in the telecamera, gets different fault value of characteristic feature of head and hand and then chooses the most suitabl fault value to be recognized for decision tree and at last realizes human-computer interaction by taking window controll as input system taking the place of mouse and keyboard in the game. .The above all build a new digital entertainment system with the specialty of human-computer Interaction.
引文
[1]董士海.人机交互的进展及面临的挑战.计算机辅助设计与图形学学报. Vol.16.No.1. 2004年1月
[2] J. K. Aggarwal, Q. Cai. Human Motion Analysis: A Review. Computer Vision and Image Understanding, Vol. 73. No. 3. 1999
[3] Ying Wu. Thomas S. Huang, "Human Hand Modeling, Analysis and Animation in the Context of Human Computer Interaction", IEEE Signal Processing Magazine, Special issue on Immersive Interactive Technology,2001,18(3): pp.51-60
[4] Bruderlin A. Calvert T W. Goal-directed, Dynamic animation of human walking. Computer Graphics,1989,23(3):19-24
[5] Ralf Plankers, Pascal Fua. Articulated Soft Objects for Multiview Shape and Motion Capture. IEEE Tran on PAMI,Vol. 25 No. 9 .2003
[6] Zoran Zivkovic, Ben Krose. An EM-like algorithm for color-histogram-based object tracking.Proc of IEEE Computer Society Conference on CVPR, 2004
[7] N. Krahnstoever, R. Sharma. Articulated Models from Video. Proc of IEEE Computer Society Conference on CVPR, 2004
[8] Jiayong Zhang, Robert Collins, Yanxi Liu. Representation and Matching of Articulated Shapes. Proc of IEEE Computer Society Conference on CVPR,2004
[9] Kentaro Toyama, Andrew Blake. Probabilistic Tracking in a Metric Space. The 8thIEEE International Conference on Computer Vision, 2001
[10] Ioannis Kakadiaris, Dimitris Metaxas.: Model-Based Estimation of 3D Human Motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22. NO. 12.2000
[11] Cheng Chang, Chung-Lin Huang.: The model-based human body motion analysis system. Image and Vision Computing, 2000
[12] Christopher R. Wren, Brian P. Clarkson, Alex P. Pentland. Understanding Purposeful Human Motion. Fourth IEEE International Conference on Automatic Face and Gesture Recognition,2000
[13] Chris Stauffer, W.E.L Grimson. Adaptive background mixture models for real time tracking. Proc of IEEE Computer Society Conference on CVPR, 1999
[14] D. Hogg, Model-based Vision: A Program to See a Walking Person, Image and Vision Computing,1983, 1(1): 5-20
[15] K. Rohr, Human Movement Analysis Based on Explicit Motion Model, Chap. 8, pp.171-198, Kluwer Academic, Dordrecht, Boston, 1997
[16] D.M. Gavrila and L.S. Davis, a 3-D Model-based Tracking of Humans in Action: a Multi-view Approach, Proc. of Conference on Computer Vision and Pattern Recognition, San Francisco, CA, 1996
[17] Wachter S, Nagel H-H. Tracking persons in monocular image sequences. Computer Vision and Image Understanding, 1999, 74 (3) : 174-192
[18] Y. Kameda, M. Minoh, and K. Ikeda, Three Dimensional Pose Estimation of an Articulated Object from its Silhouette Image, Proc. of Asian Conference on Computer Vision, 1996
[19] C. Hu, Q. Tu, Y. Li, and S. Ma, Extraction of Parameteric Human Model for Posture Recognition Using Genetic Algorithm, Proc. of the 4th International Conference on Automatic Face and Gesture Recognition, Grenoble, France, Mar. 2000
[20] M. Kass, A. Witkin, and D. Terzopoulos, Snakes: Active Contour Models, International Journal of Computer Vision (IJCV),1988 ,76(2): 321-331
[21] D. Meyer, J. Denzler, and H. Niemann, Model Based Extraction of Articulated Objects in Image Sequences, Proc. of the 4th International Conference on Image Processing, 1997
[22] Q. Delamarre and O. Faugeras, 3D Articulated Models and Multi-View Tracking with Physical Forces, Computer Vision and Image Understanding,2001,81(4): 328-357
[23] Z. Chen, H. J. Lee. Knowledge guided visual perception of 3D human gait from a single image sequence. IEEE Trans. on Systems, Man, and Cybernetics, 1992
[24] C.J. Taylor, Reconstruction of Articulated Objects from Point Correspondences in a Single Uncalibrated Image, 1999
[25] J. Gao and J. Shi, Inferring Human Upper Body Motion, submitted to International Conference on Computer Vision and Pattern Recognition, 2003
[26] Christopher Richard Wren, Ali Azarbayejani. Pfinder: Real-Time Tracking of the Human Body. IEEE Trans on PAMI, Vol. 19, No. 7, July 1997
[27] F. Lerasle, G. Rives, and M. Dhome, Tracking of Human Limbs by Multiocular Vision, Computer Vision and Image Understanding,1999, 75(2):229-246
[28] H. Sidenbladh, F. De la Torre, and M.J. Black, A Framework for Modeling the Appearance of 3D Articulated Figures, Proc. of the Fourth International Conference on Automatic Faceand Gesture Recognition, Genoble, France March 2000
[29] A. Utsumi and N. Tetsutani, Texture Adaptation for Human Tracking Using Statistical Shape Model, Proc. of International Conference on Pattern Recognition, 2002
[30] A. Lipton, H. Fujiyoshi and R. Patil, Moving Target Classification and Tracking from Real-Time Video, Proc. of IEEE Workshop on Applications of Computer Vision, pp.8-14,Princeton, NJ, 1998
[31] I. Haritaoglu, D. Harwood and L. Davis, W4: real-time surveillance of people and their activities, IEEE Trans Pattern Analysis and Machine Intelligence,2000,22 (8):809-830
[32] T. Darrell, G. Gordon, M. Harville, and J. Woodfill. Integrated person tracking using stereo, color, and pattern detection. International Journal of Computer Vision, 2000
[33] Isard M, Blake A. Condensation——conditional density propagation for visual tracking. International Journal of Computer Vision, 1998, 29 (1) : 5-28
[34] Paul A. Viola, Michael J. Jones: Robust Real-Time Face Detection. International Journal of Computer Vision USA:IEEE, 2004, 57(2):137-154
[35] Rainer Lienhart, Jochen Maydt. An Extended Set of Haar-like Features for Rapid Object Detection. IEEE ICIP 2002, Vol. 1, Sep. 2002
[36]罗忠祥,庄越挺,潘云鹤,刘丰,视频中不完全运动特征的跟踪算法,计算机辅助设计与图形学学报,2003,15(6):43-47
[37]徐一华,李京峰,贾云得,人体三维运动实时跟踪与建模系统,自动化学报,2006,9(2):31-35
[38] Erich Gamma, Richard Helm, Ralph Johnson, John Vlissides. Design Patterns: Elements of Reusable Object-Oriented software. Addison Wesley/Pearson press,October.1994
[2] J. K. Aggarwal, Q. Cai. Human Motion Analysis: A Review. Computer Vision and Image Understanding, Vol. 73. No. 3. 1999
[3] Ying Wu. Thomas S. Huang, "Human Hand Modeling, Analysis and Animation in the Context of Human Computer Interaction", IEEE Signal Processing Magazine, Special issue on Immersive Interactive Technology,2001,18(3): pp.51-60
[4] Bruderlin A. Calvert T W. Goal-directed, Dynamic animation of human walking. Computer Graphics,1989,23(3):19-24
[5] Ralf Plankers, Pascal Fua. Articulated Soft Objects for Multiview Shape and Motion Capture. IEEE Tran on PAMI,Vol. 25 No. 9 .2003
[6] Zoran Zivkovic, Ben Krose. An EM-like algorithm for color-histogram-based object tracking.Proc of IEEE Computer Society Conference on CVPR, 2004
[7] N. Krahnstoever, R. Sharma. Articulated Models from Video. Proc of IEEE Computer Society Conference on CVPR, 2004
[8] Jiayong Zhang, Robert Collins, Yanxi Liu. Representation and Matching of Articulated Shapes. Proc of IEEE Computer Society Conference on CVPR,2004
[9] Kentaro Toyama, Andrew Blake. Probabilistic Tracking in a Metric Space. The 8thIEEE International Conference on Computer Vision, 2001
[10] Ioannis Kakadiaris, Dimitris Metaxas.: Model-Based Estimation of 3D Human Motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22. NO. 12.2000
[11] Cheng Chang, Chung-Lin Huang.: The model-based human body motion analysis system. Image and Vision Computing, 2000
[12] Christopher R. Wren, Brian P. Clarkson, Alex P. Pentland. Understanding Purposeful Human Motion. Fourth IEEE International Conference on Automatic Face and Gesture Recognition,2000
[13] Chris Stauffer, W.E.L Grimson. Adaptive background mixture models for real time tracking. Proc of IEEE Computer Society Conference on CVPR, 1999
[14] D. Hogg, Model-based Vision: A Program to See a Walking Person, Image and Vision Computing,1983, 1(1): 5-20
[15] K. Rohr, Human Movement Analysis Based on Explicit Motion Model, Chap. 8, pp.171-198, Kluwer Academic, Dordrecht, Boston, 1997
[16] D.M. Gavrila and L.S. Davis, a 3-D Model-based Tracking of Humans in Action: a Multi-view Approach, Proc. of Conference on Computer Vision and Pattern Recognition, San Francisco, CA, 1996
[17] Wachter S, Nagel H-H. Tracking persons in monocular image sequences. Computer Vision and Image Understanding, 1999, 74 (3) : 174-192
[18] Y. Kameda, M. Minoh, and K. Ikeda, Three Dimensional Pose Estimation of an Articulated Object from its Silhouette Image, Proc. of Asian Conference on Computer Vision, 1996
[19] C. Hu, Q. Tu, Y. Li, and S. Ma, Extraction of Parameteric Human Model for Posture Recognition Using Genetic Algorithm, Proc. of the 4th International Conference on Automatic Face and Gesture Recognition, Grenoble, France, Mar. 2000
[20] M. Kass, A. Witkin, and D. Terzopoulos, Snakes: Active Contour Models, International Journal of Computer Vision (IJCV),1988 ,76(2): 321-331
[21] D. Meyer, J. Denzler, and H. Niemann, Model Based Extraction of Articulated Objects in Image Sequences, Proc. of the 4th International Conference on Image Processing, 1997
[22] Q. Delamarre and O. Faugeras, 3D Articulated Models and Multi-View Tracking with Physical Forces, Computer Vision and Image Understanding,2001,81(4): 328-357
[23] Z. Chen, H. J. Lee. Knowledge guided visual perception of 3D human gait from a single image sequence. IEEE Trans. on Systems, Man, and Cybernetics, 1992
[24] C.J. Taylor, Reconstruction of Articulated Objects from Point Correspondences in a Single Uncalibrated Image, 1999
[25] J. Gao and J. Shi, Inferring Human Upper Body Motion, submitted to International Conference on Computer Vision and Pattern Recognition, 2003
[26] Christopher Richard Wren, Ali Azarbayejani. Pfinder: Real-Time Tracking of the Human Body. IEEE Trans on PAMI, Vol. 19, No. 7, July 1997
[27] F. Lerasle, G. Rives, and M. Dhome, Tracking of Human Limbs by Multiocular Vision, Computer Vision and Image Understanding,1999, 75(2):229-246
[28] H. Sidenbladh, F. De la Torre, and M.J. Black, A Framework for Modeling the Appearance of 3D Articulated Figures, Proc. of the Fourth International Conference on Automatic Faceand Gesture Recognition, Genoble, France March 2000
[29] A. Utsumi and N. Tetsutani, Texture Adaptation for Human Tracking Using Statistical Shape Model, Proc. of International Conference on Pattern Recognition, 2002
[30] A. Lipton, H. Fujiyoshi and R. Patil, Moving Target Classification and Tracking from Real-Time Video, Proc. of IEEE Workshop on Applications of Computer Vision, pp.8-14,Princeton, NJ, 1998
[31] I. Haritaoglu, D. Harwood and L. Davis, W4: real-time surveillance of people and their activities, IEEE Trans Pattern Analysis and Machine Intelligence,2000,22 (8):809-830
[32] T. Darrell, G. Gordon, M. Harville, and J. Woodfill. Integrated person tracking using stereo, color, and pattern detection. International Journal of Computer Vision, 2000
[33] Isard M, Blake A. Condensation——conditional density propagation for visual tracking. International Journal of Computer Vision, 1998, 29 (1) : 5-28
[34] Paul A. Viola, Michael J. Jones: Robust Real-Time Face Detection. International Journal of Computer Vision USA:IEEE, 2004, 57(2):137-154
[35] Rainer Lienhart, Jochen Maydt. An Extended Set of Haar-like Features for Rapid Object Detection. IEEE ICIP 2002, Vol. 1, Sep. 2002
[36]罗忠祥,庄越挺,潘云鹤,刘丰,视频中不完全运动特征的跟踪算法,计算机辅助设计与图形学学报,2003,15(6):43-47
[37]徐一华,李京峰,贾云得,人体三维运动实时跟踪与建模系统,自动化学报,2006,9(2):31-35
[38] Erich Gamma, Richard Helm, Ralph Johnson, John Vlissides. Design Patterns: Elements of Reusable Object-Oriented software. Addison Wesley/Pearson press,October.1994