使用视觉注意和多特征融合的手势检测与识别
|
摘要
手势检测和识别在手语识别和人机交互中具有广泛而重要的应用.提出一种新颖的手势检测和手势识别方法.该检测方法是基于视觉注意机制检测手势,其集成了多尺度全局区域颜色、纹理、运动和背景对比度,在此基础上融合了对象性度量先验.其次,使用显著性、轮廓、局部二进制模式和梯度特征表示手势,然后采用基于度量的空间加权相似性方法融合上述4种特征和空间坐标用于确定两张图像间的相似性.最后,使用Ranking-Support Vector Machine分类器识别手势.本文设计了两个实验,一个是用于验证基于视觉注意的手势检测,另一个是用于手势识别.前者结果表明提出的手势检测方法能准确地的检测出手,能处理光照的变化和不均匀的光照.后者表明本文提出的手势识别方法优于线性和非线性方法,其识别率为96.84%,比线性和非线性回归方法分别高5.16%和2.53%.
Hand gesture detection and recognition has important applications in sign language and human-machine interfaces. In this article,a novel hand gesture detection and recognition approach is proposed. The method is based on the use of visual attention mechanism for the detection of hands,which integrates the multi-scale global regional color,texture and motion contrast and background contrast with objectness measure prior. In addition,saliency,silhouette,local binary pattern and gradient magnitude features are used to present the hand gesture and then the four features are fused with spatial coordinates to jointly determine the similarity of two images using a metricbased spatially weighed similarity. Finally,hand gesture is classified using Ranking-Support Vector Machines( Ranking-SVM) classifiers. In our paper,two experiments are conducted,one is for attention based hand gesture detection and the other is for hand gesture recognition. The former shows that the proposed hand gesture detection can demonstrate hands accurately and can deal with illumination changes and uneven illumination. The latter shows that the hand gesture recognition method outperforms the linear and non-linear methods and the average recognition precision is 96. 84%,which is 5. 16% and 2. 53% higher than linear and non-linear regression methods,respectively.
Hand gesture detection and recognition has important applications in sign language and human-machine interfaces. In this article,a novel hand gesture detection and recognition approach is proposed. The method is based on the use of visual attention mechanism for the detection of hands,which integrates the multi-scale global regional color,texture and motion contrast and background contrast with objectness measure prior. In addition,saliency,silhouette,local binary pattern and gradient magnitude features are used to present the hand gesture and then the four features are fused with spatial coordinates to jointly determine the similarity of two images using a metricbased spatially weighed similarity. Finally,hand gesture is classified using Ranking-Support Vector Machines( Ranking-SVM) classifiers. In our paper,two experiments are conducted,one is for attention based hand gesture detection and the other is for hand gesture recognition. The former shows that the proposed hand gesture detection can demonstrate hands accurately and can deal with illumination changes and uneven illumination. The latter shows that the hand gesture recognition method outperforms the linear and non-linear methods and the average recognition precision is 96. 84%,which is 5. 16% and 2. 53% higher than linear and non-linear regression methods,respectively.
引文
[1]Lamberti Luigi,Camastra Francesco.Real-time hand gesture recognition using a color glove[C].International Conference on Image Analysis and Processing,2011:365-373.
[2]Yao Yi,Li Chang-tsun.A framework for real-time hand gesture recognition in uncontrolled environments with partition matrix model based on hidden conditional random fields[C].IEEE International Conference on Systems,Man,and Cybernetics(SMC),2013:1205-1210.
[3]Angelopoulou Anastassia,Jose García-rodriguez,Psarrou Alexandra.Learning 2 D hand shapes using the topology preserving model GNG[C].9 th European Conference on Computer Vision,2006:313-324.
[4]Wan Jun,Ruan Qiu-qi,Li Wei,et al.One-shot learning gesture recognition from RGB-D data using bag of features[J].The Journal of Machine Learning Research,2013,4(1):2549-2582.
[5]Shen Xiao-hui,Gang Hua,Lance Williams,et al.Dynamic hand gesture recognition:an exemplar-based approach from motion divergence fields[J].Image and Vision Computing,2012,30(3):227-235.
[6]Keskin Cem,Kirac Furkan,Kara Yunus,et al.Real time hand pose estimation using depth image[C].IEEE International Conference on Computer Vision Workshops,2011:1228-1234.
[7]Ajallooeian Mostafa,Borji Ali,Araabi Babak-nadjar,et al.Fast hand gesture recognition based on saliency maps:An application to interactive robotic marionette playing[C].In:The 18th IEEE International Symposium on Robot and Human Interactive Communication,2009:841-847.
[8]Pisharady Pramod-kumar,Vadakkepat Prahlad,Loh Ai-poh.Attention based detection and recognition of hand postures against complex backgrounds[J].International Journal of Computer Vision,2013,101(3):403-419.
[9]Just Agnes,Marcel Sebastien.A comparative study of two state-oftheart sequence processing techniques for hand gesture recognition[J].Computer Vision and Image Understanding,2009,113(4):532-543.
[10]Pisharady Pramod-kumar,Vadakkepat Prahlad,Loh Ai-poh.Graph matching based hand posture recognition using neuro-biologically inspired features[C].In International Conference on Control,Automation,Robotics and Vision(ICARCV),2010:1151-1156.
[11]Pisharady Pramod-kumar,Stephanie Quek-shu,Vadakkepat Prahlad,et al.Hand posture recognition using neuro-biologically inspired features[C].In International Conference on Computational Intelligence,Robotics and Autonomous Systems(CIRAS),2010,103:290-297.
[12]Pisharady Pramod-kumar,Vadakkepat Prahlad,Loh Ai-poh.Fuzzyrough discriminative feature selection and classification algorithm,with application to microarray and image datasets[J].Applied Soft Computing,2011,11(4):3429-3440.
[13]Kelly Daniel,Mcdonald John,Markham Charles.A person independent system for recognition of hand postures used in sign language[J].Pattern Recognition Letters,2010,31(11):1359-1368.
[14]Yang Wen-ji,Huang Wei,Duan Liang-liang.Multi-scale global regional contrast based salient region detection[J].Journal of Computational Information Systems,2014,10(10):4071-4080.
[15]Alexe Bogdan,Deselaers Thomas,Ferrari Vittorio.What is an object?[C].IEEE Computer Society Conference on Computer Vision and Pattern Recognition,2010:73-80.
[16]Jiang Peng,Ling Hai-bin,Yu Jing-yi,et al.Salient region detection by UFO:uniqueness,focusness and objectness[C].The IEEE International Conference on Computer Vision,2013:1976-1983.
[17]Felzenszwalb Pedro Felipe,Huttenlocher Daniel P.Efficient graphbased image segmentation[J].International Journal of Computer Vision,2004,59(2):167-181.
[18]Tran Du,Sorokin Alexander,Forsyth David.Human activity recognition with metric learning[C].European Conference on Computer Vision,2008:548-561.
[19]Shi Jian-bo,Malik Jitendra.Normalized cuts and image segmentation[J],IEEE Transactions on Pattern Analysis and Machine Intelligence,2000,22(8):888-905.
[20]Ng Andrew-y,Jordan Michaell,Weiss Yair.On spectral clustering:analysis and an algorithm[C].15th Annual Neural Information Processing Systems Conference,2002:64-72.
[21]Joachims Thorsten.Optimizing seach engines using clickthrough data[C].In:Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,2002:133-142.
[22]Cao Zhe,Qin Tao,Liu Tie-yan,et al.Learning to rank:from pairwise approach to listwise approach[C].Proceedings of the 24th International Conference on Machine Learning,2007:129-136.
[23]Qin Tao,Zhang Xu-dong,Tsai Ming-feng,et al.Query-level loss function for information retrieval[J].Information Processing and Management,2007,44(2):838-855.
[24]Joachims Thorsten.Svm light-an implementation of support vector machine in c[EB/OL].http://svmlight.joachims.org,2002.
[25]Shen Xiao-hui,Wu Ying.A unified approach to salient object detection via low rank matrix recovery[C].In IEEE Conference on Computer Vision and Pattern Recognition(CVPR),2012:853-860.
[26]Cheng Ming-ming,Zhang Guo-xin,Mitra Niloy-j,et al.Global contrast based salient region detection[C].In IEEE Conference on Computer Vision and Pattern Recognition(CVPR),2011:409-416.
[27]Achanta Radhakrishna,Hemami Sheila,Estrada Francisco.Frequency-tuned salient region detection[C].In IEEE Conference on Computer Vision and Pattern Recognition,2009:1597-1604.
[28]Kim Tae-Kyun,Wong Kwan-Yee Kenneth,Cipolla Roberto.Tensor canonical correlation analysis for action classification[C].In IEEE Conference on Computer Vision and Pattern Recognition,2007:1-8.
[29]Chung Kai-min,Kao Wei-chun,Sun Chia-Liang,et al.Radius margin bounds for support vector machines with the RBF kernel[J].Neural Computation,2003,15(11):2643-2681.
[2]Yao Yi,Li Chang-tsun.A framework for real-time hand gesture recognition in uncontrolled environments with partition matrix model based on hidden conditional random fields[C].IEEE International Conference on Systems,Man,and Cybernetics(SMC),2013:1205-1210.
[3]Angelopoulou Anastassia,Jose García-rodriguez,Psarrou Alexandra.Learning 2 D hand shapes using the topology preserving model GNG[C].9 th European Conference on Computer Vision,2006:313-324.
[4]Wan Jun,Ruan Qiu-qi,Li Wei,et al.One-shot learning gesture recognition from RGB-D data using bag of features[J].The Journal of Machine Learning Research,2013,4(1):2549-2582.
[5]Shen Xiao-hui,Gang Hua,Lance Williams,et al.Dynamic hand gesture recognition:an exemplar-based approach from motion divergence fields[J].Image and Vision Computing,2012,30(3):227-235.
[6]Keskin Cem,Kirac Furkan,Kara Yunus,et al.Real time hand pose estimation using depth image[C].IEEE International Conference on Computer Vision Workshops,2011:1228-1234.
[7]Ajallooeian Mostafa,Borji Ali,Araabi Babak-nadjar,et al.Fast hand gesture recognition based on saliency maps:An application to interactive robotic marionette playing[C].In:The 18th IEEE International Symposium on Robot and Human Interactive Communication,2009:841-847.
[8]Pisharady Pramod-kumar,Vadakkepat Prahlad,Loh Ai-poh.Attention based detection and recognition of hand postures against complex backgrounds[J].International Journal of Computer Vision,2013,101(3):403-419.
[9]Just Agnes,Marcel Sebastien.A comparative study of two state-oftheart sequence processing techniques for hand gesture recognition[J].Computer Vision and Image Understanding,2009,113(4):532-543.
[10]Pisharady Pramod-kumar,Vadakkepat Prahlad,Loh Ai-poh.Graph matching based hand posture recognition using neuro-biologically inspired features[C].In International Conference on Control,Automation,Robotics and Vision(ICARCV),2010:1151-1156.
[11]Pisharady Pramod-kumar,Stephanie Quek-shu,Vadakkepat Prahlad,et al.Hand posture recognition using neuro-biologically inspired features[C].In International Conference on Computational Intelligence,Robotics and Autonomous Systems(CIRAS),2010,103:290-297.
[12]Pisharady Pramod-kumar,Vadakkepat Prahlad,Loh Ai-poh.Fuzzyrough discriminative feature selection and classification algorithm,with application to microarray and image datasets[J].Applied Soft Computing,2011,11(4):3429-3440.
[13]Kelly Daniel,Mcdonald John,Markham Charles.A person independent system for recognition of hand postures used in sign language[J].Pattern Recognition Letters,2010,31(11):1359-1368.
[14]Yang Wen-ji,Huang Wei,Duan Liang-liang.Multi-scale global regional contrast based salient region detection[J].Journal of Computational Information Systems,2014,10(10):4071-4080.
[15]Alexe Bogdan,Deselaers Thomas,Ferrari Vittorio.What is an object?[C].IEEE Computer Society Conference on Computer Vision and Pattern Recognition,2010:73-80.
[16]Jiang Peng,Ling Hai-bin,Yu Jing-yi,et al.Salient region detection by UFO:uniqueness,focusness and objectness[C].The IEEE International Conference on Computer Vision,2013:1976-1983.
[17]Felzenszwalb Pedro Felipe,Huttenlocher Daniel P.Efficient graphbased image segmentation[J].International Journal of Computer Vision,2004,59(2):167-181.
[18]Tran Du,Sorokin Alexander,Forsyth David.Human activity recognition with metric learning[C].European Conference on Computer Vision,2008:548-561.
[19]Shi Jian-bo,Malik Jitendra.Normalized cuts and image segmentation[J],IEEE Transactions on Pattern Analysis and Machine Intelligence,2000,22(8):888-905.
[20]Ng Andrew-y,Jordan Michaell,Weiss Yair.On spectral clustering:analysis and an algorithm[C].15th Annual Neural Information Processing Systems Conference,2002:64-72.
[21]Joachims Thorsten.Optimizing seach engines using clickthrough data[C].In:Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,2002:133-142.
[22]Cao Zhe,Qin Tao,Liu Tie-yan,et al.Learning to rank:from pairwise approach to listwise approach[C].Proceedings of the 24th International Conference on Machine Learning,2007:129-136.
[23]Qin Tao,Zhang Xu-dong,Tsai Ming-feng,et al.Query-level loss function for information retrieval[J].Information Processing and Management,2007,44(2):838-855.
[24]Joachims Thorsten.Svm light-an implementation of support vector machine in c[EB/OL].http://svmlight.joachims.org,2002.
[25]Shen Xiao-hui,Wu Ying.A unified approach to salient object detection via low rank matrix recovery[C].In IEEE Conference on Computer Vision and Pattern Recognition(CVPR),2012:853-860.
[26]Cheng Ming-ming,Zhang Guo-xin,Mitra Niloy-j,et al.Global contrast based salient region detection[C].In IEEE Conference on Computer Vision and Pattern Recognition(CVPR),2011:409-416.
[27]Achanta Radhakrishna,Hemami Sheila,Estrada Francisco.Frequency-tuned salient region detection[C].In IEEE Conference on Computer Vision and Pattern Recognition,2009:1597-1604.
[28]Kim Tae-Kyun,Wong Kwan-Yee Kenneth,Cipolla Roberto.Tensor canonical correlation analysis for action classification[C].In IEEE Conference on Computer Vision and Pattern Recognition,2007:1-8.
[29]Chung Kai-min,Kao Wei-chun,Sun Chia-Liang,et al.Radius margin bounds for support vector machines with the RBF kernel[J].Neural Computation,2003,15(11):2643-2681.