面向人脸特征点定位和姿态估计任务协同的DCNN方法
|
摘要
为了提高复杂背景下面部信息的识别性能,提出了一种面向人脸特征点定位和姿态估计任务协同的深度卷积神经网络(DCNN)方法。首先从视频图像中检测出人脸信息;其次设计一个深度卷积网络模型,将人脸特征点定位和姿态估计两个任务协同优化,同时回归得到人脸特征点坐标和姿态角度值,然后融合生成相应的人机交互信息;最后采用公开数据集和实际场景数据进行测试,并与其他现有方法进行比对分析。实验结果表明:该方法在人脸特征点定位和姿态估计上表现出较好的性能,在光照变化、表情变化、部分遮挡等复杂条件下人机交互应用也取得了良好的准确性和鲁棒性,平均处理速度约16帧/s,具备一定的实用性。
In order to improve the performance of facial information recognition under complex conditions,a novel deep convolutional neural network( DCNN) method is proposed for the task coordination of facial landmark localization and head pose estimation. First,the facial information is detected from video images. Secondly,a DCNN model is designed for synergistic optimization of both facial landmark localization and head pose estimation tasks,and then used to simultaneously estimate the coordinates of facial landmarks and the angles of head pose,which are then fused to generate the humancomputer interaction information. Finally,the proposed method is tested on both public datasets and actual scene data,and then compared with other state of the art methods. Experimental results show that the proposed approach performs better in facial landmark localization and pose estimation,and also achieves good accuracy and robustness in the HCI applications under complex conditions of illumination variations,expression changes and partial collusions,with the average speed at 16 frames per second,which demonstrates its efficiency and practicality.
In order to improve the performance of facial information recognition under complex conditions,a novel deep convolutional neural network( DCNN) method is proposed for the task coordination of facial landmark localization and head pose estimation. First,the facial information is detected from video images. Secondly,a DCNN model is designed for synergistic optimization of both facial landmark localization and head pose estimation tasks,and then used to simultaneously estimate the coordinates of facial landmarks and the angles of head pose,which are then fused to generate the humancomputer interaction information. Finally,the proposed method is tested on both public datasets and actual scene data,and then compared with other state of the art methods. Experimental results show that the proposed approach performs better in facial landmark localization and pose estimation,and also achieves good accuracy and robustness in the HCI applications under complex conditions of illumination variations,expression changes and partial collusions,with the average speed at 16 frames per second,which demonstrates its efficiency and practicality.
引文
[1] Xu X,Zhang Y,Zhang S,et al. 3D hand gesture tracking and recognition for controlling an intelligent wheelchair[J]. International Journal of Computer Applications in Technology,2014,49(2):104-112.
[2] Veneri G,Federighi P,Rosini F,et al. Influences of data filtering on human-computer interaction by gazecontingent display and eye-tracking applications[J].Computers in Human Behavior,2010,26(6):1 555-1 563.
[3] Murphy-Chutorian E,Trivedi M M. Head pose estimation in computer vision:a survey[J]. IEEE Transactions on Pattern Analysis&Machine Intelligence,2009,31(4):607-626.
[4] Asthana A,Zafeiriou S,Cheng S,et al. Robust discriminative response map fitting with constrained local models[C]//Proceedings of the IEEE Conference on Computer Vision&Pattern Recognition. Portland,OR,USA,2013,3444-3451.
[5] Tzimiropoulos G,Pantic M. Gauss-newton deformable part models for face alignment in-the-wild[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus,OH,USA,2014,1851-1858.
[6] Baltruaitis T,Robinson P,Morency L P. Continuous conditional neural fields for structured regression[C]//Proceedings of Computer Vision–ECCV2014. Zurich,Switzerland,2014,593-608.
[7] Ren S,Cao X,Wei Y,et al. Face alignment at 3000 fps via regressing local binary features[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus,OH,USA,2014,1685-1692.
[8] Sun Y,Wang X,Tang X. Deep convolutional network cascade for facial point detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Portland,OR,USA,2013,3476-3483.
[9]程淑红,刘洁.基于多尺度韦伯脸和梯度脸的复杂光照下人脸识别研究[J].计量学报,2017,38(1):60-64.Cheng S,Liu J. Face Recognition under Complex ILLumination Based on Multi-scale Weber-face and Gradient-face[J]. Acta Metrologica Sinica,2017,38(1):60-64.
[10]唐云祁,孙哲南,谭铁牛.头部姿势估计研究综述[J].模式识别与人工智能,2014,27(3):213-225.Tang Y,Sun Z,Tan T. A Survey on Head Pose Estimation[J]. Pattern Recognition and Artificial Intelligence,2014,27(3):213-225.
[11] Ma B,Chai X,Wang T. A novel feature descriptor based on biologically inspired feature for head pose estimation[J]. Neurocomputing,2013,115:1-10.
[12] Foytik J,Asari V K. A Two-Layer framework for piecewise linear manifold-based head pose estimation[J]. International Journal of Computer Vision,2013,101(2):270-287.
[13] Lu J,Tan Y P. Ordinary preserving manifold analysis for human age and head pose estimation[J]. IEEE Transactions on Human-Machine Systems,2013,43(2):249-258.
[14] Lécun Y,Bottou L,Bengio Y,et al. Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[15] Le Cun Y,Bengio Y,Hinton G. Deep learning[J].Nature,2015,521(7553):436-444.
[16] Krizhevsky A,Sutskever I,Hinton G E. Image Net classification with deep convolutional neural networks[C]//Proceedings of Advances in Neural Information Processing Systems,. Lake Tahoe,Nevada,USA,2012:1097-1105.
[17] Sermanet P,Eigen D,Zhang X,et al. Overfeat:Integrated recognition,localization and detection using convolutional networks[J]. Eprint Arxiv,2013.
[18] Sun Y,Wang X,Tang X. Deep Convolutional network cascade for facial point detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition,Portland,OR,USA,2013:3476-3483.
[19]赵一中,刘文波.基于深度信念网络的非限制性人脸识别算法研究[J].计量学报,2017,38(1):65-68.Zhao Y,Liu W. Research on Unconstrained Face Recognition Based on DBNs Network[J]. Acta Metrologica Sinica,2017,38(1):65-68.
[20] Margarita Osadchy Y L C,Miller M L. Synergistic face detection and pose estimation with energy-based models[J]. Journal of Machine Learning Research,2007,8(1):1017-1024.
[21] Zhang Z,Luo P,Chen C L,et al. Facial Landmark Detection by Deep Multi-task Learning[C]//Proceedings of European Conference on Computer Vision,Columbus,OH,USA,2014:94-108.
[22] Simonyan K,Zisserman A. Very deep convolutional networks for large-scale image recognition[J/OL].(2015-04-10)[2017-07-01]. https://arxiv. org/abs/1409. 1556
[23] Sagonas C,Tzimiropoulos G,Zafeiriou S,et al. 300 faces in-the-wild challenge:The first facial landmark localization challenge[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops. Sydney,Nsw,Australia,2013:397-403.
[24] Wei L,Lee E J. Improved Multi-pose 2D Face Recognition Using 3D Face Model with Camera Pose Estimation Approach and n D-PCA Recognition Algorithm[C]//Proceedings of the Third IEEE International Conference on Convergence and Hybrid Information Technology. Busan,South,Korea,2008,2:728-736.
[25] Jain V,Learned-Miller E. FDDB:A benchmark for face detection in unconstrained settings[R]. UMass Amherst Technical Report,2010.
[26] Jia Y,Shelhamer E, Donahue J, et al. Caffe:Convolutional architecture for fast feature embedding[C]//Proceedings of the ACM International Conference on Multimedia. Orlando,Florida,USA,2014:675-678.
[27] Yang H,Jia X,Loy C C,et al. An empirical study of recent face alignment methods[J/OL].(2015-11-16)[2017-07-01]. https://arxiv. org/abs/1511. 05049.
[28] Yang X X,Torre F D L. Supervised Descent Method and Its Applications to Face Alignment[C]//IEEE Conference on Computer Vision and Pattern Recognition. Portland,OR,USA,2013:532-539.
[2] Veneri G,Federighi P,Rosini F,et al. Influences of data filtering on human-computer interaction by gazecontingent display and eye-tracking applications[J].Computers in Human Behavior,2010,26(6):1 555-1 563.
[3] Murphy-Chutorian E,Trivedi M M. Head pose estimation in computer vision:a survey[J]. IEEE Transactions on Pattern Analysis&Machine Intelligence,2009,31(4):607-626.
[4] Asthana A,Zafeiriou S,Cheng S,et al. Robust discriminative response map fitting with constrained local models[C]//Proceedings of the IEEE Conference on Computer Vision&Pattern Recognition. Portland,OR,USA,2013,3444-3451.
[5] Tzimiropoulos G,Pantic M. Gauss-newton deformable part models for face alignment in-the-wild[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus,OH,USA,2014,1851-1858.
[6] Baltruaitis T,Robinson P,Morency L P. Continuous conditional neural fields for structured regression[C]//Proceedings of Computer Vision–ECCV2014. Zurich,Switzerland,2014,593-608.
[7] Ren S,Cao X,Wei Y,et al. Face alignment at 3000 fps via regressing local binary features[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus,OH,USA,2014,1685-1692.
[8] Sun Y,Wang X,Tang X. Deep convolutional network cascade for facial point detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Portland,OR,USA,2013,3476-3483.
[9]程淑红,刘洁.基于多尺度韦伯脸和梯度脸的复杂光照下人脸识别研究[J].计量学报,2017,38(1):60-64.Cheng S,Liu J. Face Recognition under Complex ILLumination Based on Multi-scale Weber-face and Gradient-face[J]. Acta Metrologica Sinica,2017,38(1):60-64.
[10]唐云祁,孙哲南,谭铁牛.头部姿势估计研究综述[J].模式识别与人工智能,2014,27(3):213-225.Tang Y,Sun Z,Tan T. A Survey on Head Pose Estimation[J]. Pattern Recognition and Artificial Intelligence,2014,27(3):213-225.
[11] Ma B,Chai X,Wang T. A novel feature descriptor based on biologically inspired feature for head pose estimation[J]. Neurocomputing,2013,115:1-10.
[12] Foytik J,Asari V K. A Two-Layer framework for piecewise linear manifold-based head pose estimation[J]. International Journal of Computer Vision,2013,101(2):270-287.
[13] Lu J,Tan Y P. Ordinary preserving manifold analysis for human age and head pose estimation[J]. IEEE Transactions on Human-Machine Systems,2013,43(2):249-258.
[14] Lécun Y,Bottou L,Bengio Y,et al. Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[15] Le Cun Y,Bengio Y,Hinton G. Deep learning[J].Nature,2015,521(7553):436-444.
[16] Krizhevsky A,Sutskever I,Hinton G E. Image Net classification with deep convolutional neural networks[C]//Proceedings of Advances in Neural Information Processing Systems,. Lake Tahoe,Nevada,USA,2012:1097-1105.
[17] Sermanet P,Eigen D,Zhang X,et al. Overfeat:Integrated recognition,localization and detection using convolutional networks[J]. Eprint Arxiv,2013.
[18] Sun Y,Wang X,Tang X. Deep Convolutional network cascade for facial point detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition,Portland,OR,USA,2013:3476-3483.
[19]赵一中,刘文波.基于深度信念网络的非限制性人脸识别算法研究[J].计量学报,2017,38(1):65-68.Zhao Y,Liu W. Research on Unconstrained Face Recognition Based on DBNs Network[J]. Acta Metrologica Sinica,2017,38(1):65-68.
[20] Margarita Osadchy Y L C,Miller M L. Synergistic face detection and pose estimation with energy-based models[J]. Journal of Machine Learning Research,2007,8(1):1017-1024.
[21] Zhang Z,Luo P,Chen C L,et al. Facial Landmark Detection by Deep Multi-task Learning[C]//Proceedings of European Conference on Computer Vision,Columbus,OH,USA,2014:94-108.
[22] Simonyan K,Zisserman A. Very deep convolutional networks for large-scale image recognition[J/OL].(2015-04-10)[2017-07-01]. https://arxiv. org/abs/1409. 1556
[23] Sagonas C,Tzimiropoulos G,Zafeiriou S,et al. 300 faces in-the-wild challenge:The first facial landmark localization challenge[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops. Sydney,Nsw,Australia,2013:397-403.
[24] Wei L,Lee E J. Improved Multi-pose 2D Face Recognition Using 3D Face Model with Camera Pose Estimation Approach and n D-PCA Recognition Algorithm[C]//Proceedings of the Third IEEE International Conference on Convergence and Hybrid Information Technology. Busan,South,Korea,2008,2:728-736.
[25] Jain V,Learned-Miller E. FDDB:A benchmark for face detection in unconstrained settings[R]. UMass Amherst Technical Report,2010.
[26] Jia Y,Shelhamer E, Donahue J, et al. Caffe:Convolutional architecture for fast feature embedding[C]//Proceedings of the ACM International Conference on Multimedia. Orlando,Florida,USA,2014:675-678.
[27] Yang H,Jia X,Loy C C,et al. An empirical study of recent face alignment methods[J/OL].(2015-11-16)[2017-07-01]. https://arxiv. org/abs/1511. 05049.
[28] Yang X X,Torre F D L. Supervised Descent Method and Its Applications to Face Alignment[C]//IEEE Conference on Computer Vision and Pattern Recognition. Portland,OR,USA,2013:532-539.