模板匹配的三维手势识别算法
|
摘要
针对三维空间的手势识别,基于轻量级的二维平面手势识别算法进行了扩展及改进,将其扩展到了三维空间,提出了基于投影的单次旋转模板匹配识别方法.将手势模板集划分为空间平面手势和空间立体手势2类模板;对于空间平面手势模板,输入手势轨迹需要先投影到相应的坐标平面,再进入模板匹配流程;基于矩阵奇异值分解,计算归一化手势轨迹与模板之间的最优旋转矩阵,经过一次刚体转置即可完成与模板的匹配度计算.实验结果表明,该方法对三维手势具有较高的识别率,且对于空间平面手势、空间立体手势的识别能力较为均衡.
This paper extended and improved the algorithm, a lightweight planar 2D hand gesture recognition algorithm, and realized a 3D gesture recognition algorithm based on projection and single rotation. The gesture template library was divided into two types: spatial planar gesture and spatial stereo gesture. For the templates of the spatial planar gesture, the input gesture trajectory needed to be projected onto the corresponding coordinate plane before the template matching. Based on the matrix singular value decomposition method, the optimal transposing rotation matrix between the normalized input gesture and the templates was calculated. With the transposing matrix, only a rigid transpose operation was needed for matching degree calculation of a template. Experiments show that this method has a high recognition rate for 3D gesture recognition, and the recognition ability is more balanced for 3D plane and stereo gestures.
This paper extended and improved the algorithm, a lightweight planar 2D hand gesture recognition algorithm, and realized a 3D gesture recognition algorithm based on projection and single rotation. The gesture template library was divided into two types: spatial planar gesture and spatial stereo gesture. For the templates of the spatial planar gesture, the input gesture trajectory needed to be projected onto the corresponding coordinate plane before the template matching. Based on the matrix singular value decomposition method, the optimal transposing rotation matrix between the normalized input gesture and the templates was calculated. With the transposing matrix, only a rigid transpose operation was needed for matching degree calculation of a template. Experiments show that this method has a high recognition rate for 3D gesture recognition, and the recognition ability is more balanced for 3D plane and stereo gestures.
引文
[1]Caon M,Yong Y,Tscherrig J,et al.Democratizing 3D dynamic gestures recognition[C]//Proceedings of the 1st IEEE Workshop on User-Centered Computer Vision.Los Alamitos:IEEE Computer Society Press,2013:7-12
[2]Lu Lu,Tian Feng,Dai Guozhong,et al.A study of the multimodal cognition and interaction based on touch,audition and vision[J].Journal of Computer-Aided Design&Computer Graphics,2014,26(4):654-661(in Chinese)(路璐,田丰,戴国忠,等.融合触、听、视觉的多通道认知和交互模型[J].计算机辅助设计与图形学学报,2014,26(4):654-661)
[3]Cheema S,Hoffman M,Laviola J J.3D gesture classification with linear acceleration and angular velocity sensing devices for video games[J].Entertainment Computing,2013,4(1):11-24
[4]Feng Zhiquan,Yang Bo,Xu Tao,et al.3D direct human-computer interface paradigm based on free hand tracking[J].Chinese Journal of Computers,2014,37(6):1309-1323(in Chinese)(冯志全,杨波,徐涛,等.基于自然手势跟踪的直接操作型三维人机交互范式[J].计算机学报,2014,37(6):1309-1323)
[5]Zhang K,Zhai Y,Leong H W,et al.An interaction educational computer game framework using hand gesture recognition[C]//Proceedings of the 4th International Conference on Internet Multimedia Computing and Service.New York:ACM Press,2012:219-222
[6]Vatavu R D,Zaiti I A.Leap gestures for TV:insights from an elicitation study[C]//Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video.New York:ACM Press,2014:131-138
[7]Wang C L,Gao W,Shan S G.An approach based on phonemes to large vocabulary Chinese sign language recognition[C]//Proceedings of the 5th IEEE International Conference on Automatic Face and Gesture Recognition.Los Alamitos:IEEE Computer Society Press,2002:411-416
[8]Ferracani A,Pezzatini D,Bimbo A D.A natural and immersive virtual interface for the surgical safety checklist training[C]//Proceedings of the ACM International Workshop on Serious Games.New York:ACM Press,2014:27-32
[9]Jalaliniya S,Smith J,Sousa M,et al.Touch-less interaction with medical images using hand&foot gestures[C]//Proceedings of the ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication.New York:ACM Press,2013:1265-1274
[10]Cipolla R,Hollinghurst N J.Human-robot interface by pointing with uncalibrated stereo vision[J].Image and Vision Computing,1996,14(3):171-178
[11]La Viola J J,Jr.An introduction to 3D gestural interfaces[C]//Computer Graphics Proceedings,Annual Conference Series,ACM SIGGRAPH.New York:ACM Press,2014:Article No.25
[12]Oikonomidis I,Kyriazis N,Argyros A A.Tracking the articulated motion of two strongly interacting hands[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2012:1862-1869
[13]Lee U,Tanaka J.Finger identification and hand gesture recognition techniques for natural user interface[C]//Proceedings of the 11th Asia Pacific Conference on Computer Human Interaction.New York:ACM Press,2013:274-279
[14]Chen X A,Schwarz J,Harrison C,et al.Air+touch:interweaving touch&in-air gestures[C]//Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology.New York:ACM Press,2014:519-525
[15]Wobbrock J O,Wilson A D,Li Y.Gestures without libraries,toolkits or training:a$1 recognizer for user interface prototypes[C]//Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology.New York:ACM Press,2007:159-168
[16]Bott J N,Crowley J G,La Viola J J,Jr.Exploring 3D gestural interfaces for music creation in video games[C]//Proceedings of the 4th International Conference on Foundations of Digital Games.New York:ACM Press,2009:18-25
[17]Charbonneau E,Miller A,Wingrave C,et al.Understanding visual interfaces for the next generation of dance-based rhythm video games[C]//Proceedings of the ACM SIGGRAPH Symposium on Video Games.New York:ACM Press,2009:119-126
[18]Yang Y H,Ma M D,Pang H B,et al.Dynamic hand gesture recognition using kinematic features based on hidden markov model[M]//Lecture Notes in Electrical Engineering.Heidelberg:Springer,2013,227:255-262
[19]Chung H,Yang H D.Conditional random field-based gesture recognition with depth information[J].Optical Engineering,2013,52(1):017201
[20]Keskin C,Kirac F,Kara Y E,et al.Real time hand pose estimation using depth sensors[C]//Proceedings of IEEE International Conference on Computer Vision Workshops.Los Alamitos:IEEE Computer Society Press,2011:1228-1234
[21]Kratz S,Rohs M.A$3 gesture recognizer:simple gesture recognition for devices equipped with 3D acceleration sensors[C]//Proceedings of the 15th International Conference on Intelligent User Interfaces.New York:ACM Press,2010:341-344
[22]Jiang Yingying,Tian Feng,Wang Xugang,et al.Adaptive symbol recognition for sketch-based interfaces based on template matching and SVM[J].Chinese Journal of Computers,2009,32(2):252-260(in Chinese)(姜映映,田丰,王绪刚,等.基于模板匹配和SVM的草图符号自适应识别方法[J].计算机学报,2009,32(2):252-260)
[23]Rubine D.Specifying gestures by example[C]//Proceedings of the 18th Annual Conference on Computer Graphics and Interactive Techniques.New York:ACM Press,1991:329-337
[24]Kristensson P O,Denby L C.Continuous recognition and visualization of pen strokes and touch-screen gestures[C]//Proceedings of the 8th Eurographics Symposium on SketchBased Interfaces and Modeling.New York:ACM Press,2011:95-102
[25]Kristensson P O,Nicholson T,Quigley A.Continuous recognition of one-handed and two-handed gestures using 3D full-body motion tracking sensors[C]//Proceedings of the ACM International Conference on Intelligent User Interfaces.New York:ACM Press,2012:89-92
[26]Kratz S,Rohs M.Protractor3D:a closed-form solution to rotation-invariant 3D gestures[C]//Proceedings of the 16th International Conference on Intelligent User Interfaces.New York:ACM Press,2011:371-374
[27]Klema V,Laub A J.The singular value decomposition:its computation and some applications[J].IEEE Transactions on Automatic Control,1980,25(2):164-176
[28]Sorkine O.Least-squares rigid motion using SVD[OL].[2015-08-11].http://igl.ethz.ch/projects/ARAP/svd_rot.pdf
[29]Deift P,Demmel J,Li L C,et al.The bidiagonal singular value decomposition and Hamiltonian mechanics[J].SIAM Journal on Numerical Analysis,1991,28(5):1463-1516
[30]Zou Hongxing,Wang Dianjun,Dai Qionghai,et al.Singular value decomposition for extended matrix[J].Acta Electronica Sinica,2001,29(3):289-292(in Chinese)(邹红星,王殿军,戴琼海,等.延拓矩阵的奇异值分解[J].电子学报,2001,29(3):289-292)
[31]Zhao Xuezhi,Ye Bangyan,Chen Tongjian.Multi-partition and double-direction shrink QR algorithm for singular value decomposition of large-scale matrix[J].Journal of South China University of Technology:Natural Science Edition,2010,38(1):1-8(in Chinese)(赵学智,叶邦彦,陈统坚.大型矩阵奇异值分解的多次分割双向收缩QR算法[J].华南理工大学学报:自然科学版,2010,38(1):1-8)
[32]Wang Bailing,Tian Zhihong,Zhang Yongzheng.Optimization of singular vector decomposition algorithm[J].Acta Electronica Sinica,2010,38(10):2234-2239(in Chinese)(王佰玲,田志宏,张永铮.奇异值分解算法优化[J].电子学报,2010,38(10):2234-2239)
[2]Lu Lu,Tian Feng,Dai Guozhong,et al.A study of the multimodal cognition and interaction based on touch,audition and vision[J].Journal of Computer-Aided Design&Computer Graphics,2014,26(4):654-661(in Chinese)(路璐,田丰,戴国忠,等.融合触、听、视觉的多通道认知和交互模型[J].计算机辅助设计与图形学学报,2014,26(4):654-661)
[3]Cheema S,Hoffman M,Laviola J J.3D gesture classification with linear acceleration and angular velocity sensing devices for video games[J].Entertainment Computing,2013,4(1):11-24
[4]Feng Zhiquan,Yang Bo,Xu Tao,et al.3D direct human-computer interface paradigm based on free hand tracking[J].Chinese Journal of Computers,2014,37(6):1309-1323(in Chinese)(冯志全,杨波,徐涛,等.基于自然手势跟踪的直接操作型三维人机交互范式[J].计算机学报,2014,37(6):1309-1323)
[5]Zhang K,Zhai Y,Leong H W,et al.An interaction educational computer game framework using hand gesture recognition[C]//Proceedings of the 4th International Conference on Internet Multimedia Computing and Service.New York:ACM Press,2012:219-222
[6]Vatavu R D,Zaiti I A.Leap gestures for TV:insights from an elicitation study[C]//Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video.New York:ACM Press,2014:131-138
[7]Wang C L,Gao W,Shan S G.An approach based on phonemes to large vocabulary Chinese sign language recognition[C]//Proceedings of the 5th IEEE International Conference on Automatic Face and Gesture Recognition.Los Alamitos:IEEE Computer Society Press,2002:411-416
[8]Ferracani A,Pezzatini D,Bimbo A D.A natural and immersive virtual interface for the surgical safety checklist training[C]//Proceedings of the ACM International Workshop on Serious Games.New York:ACM Press,2014:27-32
[9]Jalaliniya S,Smith J,Sousa M,et al.Touch-less interaction with medical images using hand&foot gestures[C]//Proceedings of the ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication.New York:ACM Press,2013:1265-1274
[10]Cipolla R,Hollinghurst N J.Human-robot interface by pointing with uncalibrated stereo vision[J].Image and Vision Computing,1996,14(3):171-178
[11]La Viola J J,Jr.An introduction to 3D gestural interfaces[C]//Computer Graphics Proceedings,Annual Conference Series,ACM SIGGRAPH.New York:ACM Press,2014:Article No.25
[12]Oikonomidis I,Kyriazis N,Argyros A A.Tracking the articulated motion of two strongly interacting hands[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2012:1862-1869
[13]Lee U,Tanaka J.Finger identification and hand gesture recognition techniques for natural user interface[C]//Proceedings of the 11th Asia Pacific Conference on Computer Human Interaction.New York:ACM Press,2013:274-279
[14]Chen X A,Schwarz J,Harrison C,et al.Air+touch:interweaving touch&in-air gestures[C]//Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology.New York:ACM Press,2014:519-525
[15]Wobbrock J O,Wilson A D,Li Y.Gestures without libraries,toolkits or training:a$1 recognizer for user interface prototypes[C]//Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology.New York:ACM Press,2007:159-168
[16]Bott J N,Crowley J G,La Viola J J,Jr.Exploring 3D gestural interfaces for music creation in video games[C]//Proceedings of the 4th International Conference on Foundations of Digital Games.New York:ACM Press,2009:18-25
[17]Charbonneau E,Miller A,Wingrave C,et al.Understanding visual interfaces for the next generation of dance-based rhythm video games[C]//Proceedings of the ACM SIGGRAPH Symposium on Video Games.New York:ACM Press,2009:119-126
[18]Yang Y H,Ma M D,Pang H B,et al.Dynamic hand gesture recognition using kinematic features based on hidden markov model[M]//Lecture Notes in Electrical Engineering.Heidelberg:Springer,2013,227:255-262
[19]Chung H,Yang H D.Conditional random field-based gesture recognition with depth information[J].Optical Engineering,2013,52(1):017201
[20]Keskin C,Kirac F,Kara Y E,et al.Real time hand pose estimation using depth sensors[C]//Proceedings of IEEE International Conference on Computer Vision Workshops.Los Alamitos:IEEE Computer Society Press,2011:1228-1234
[21]Kratz S,Rohs M.A$3 gesture recognizer:simple gesture recognition for devices equipped with 3D acceleration sensors[C]//Proceedings of the 15th International Conference on Intelligent User Interfaces.New York:ACM Press,2010:341-344
[22]Jiang Yingying,Tian Feng,Wang Xugang,et al.Adaptive symbol recognition for sketch-based interfaces based on template matching and SVM[J].Chinese Journal of Computers,2009,32(2):252-260(in Chinese)(姜映映,田丰,王绪刚,等.基于模板匹配和SVM的草图符号自适应识别方法[J].计算机学报,2009,32(2):252-260)
[23]Rubine D.Specifying gestures by example[C]//Proceedings of the 18th Annual Conference on Computer Graphics and Interactive Techniques.New York:ACM Press,1991:329-337
[24]Kristensson P O,Denby L C.Continuous recognition and visualization of pen strokes and touch-screen gestures[C]//Proceedings of the 8th Eurographics Symposium on SketchBased Interfaces and Modeling.New York:ACM Press,2011:95-102
[25]Kristensson P O,Nicholson T,Quigley A.Continuous recognition of one-handed and two-handed gestures using 3D full-body motion tracking sensors[C]//Proceedings of the ACM International Conference on Intelligent User Interfaces.New York:ACM Press,2012:89-92
[26]Kratz S,Rohs M.Protractor3D:a closed-form solution to rotation-invariant 3D gestures[C]//Proceedings of the 16th International Conference on Intelligent User Interfaces.New York:ACM Press,2011:371-374
[27]Klema V,Laub A J.The singular value decomposition:its computation and some applications[J].IEEE Transactions on Automatic Control,1980,25(2):164-176
[28]Sorkine O.Least-squares rigid motion using SVD[OL].[2015-08-11].http://igl.ethz.ch/projects/ARAP/svd_rot.pdf
[29]Deift P,Demmel J,Li L C,et al.The bidiagonal singular value decomposition and Hamiltonian mechanics[J].SIAM Journal on Numerical Analysis,1991,28(5):1463-1516
[30]Zou Hongxing,Wang Dianjun,Dai Qionghai,et al.Singular value decomposition for extended matrix[J].Acta Electronica Sinica,2001,29(3):289-292(in Chinese)(邹红星,王殿军,戴琼海,等.延拓矩阵的奇异值分解[J].电子学报,2001,29(3):289-292)
[31]Zhao Xuezhi,Ye Bangyan,Chen Tongjian.Multi-partition and double-direction shrink QR algorithm for singular value decomposition of large-scale matrix[J].Journal of South China University of Technology:Natural Science Edition,2010,38(1):1-8(in Chinese)(赵学智,叶邦彦,陈统坚.大型矩阵奇异值分解的多次分割双向收缩QR算法[J].华南理工大学学报:自然科学版,2010,38(1):1-8)
[32]Wang Bailing,Tian Zhihong,Zhang Yongzheng.Optimization of singular vector decomposition algorithm[J].Acta Electronica Sinica,2010,38(10):2234-2239(in Chinese)(王佰玲,田志宏,张永铮.奇异值分解算法优化[J].电子学报,2010,38(10):2234-2239)