基于关键点的由粗到精三维人脸特征点定位
|
摘要
提出了一个基于关键点由粗到精的三维人脸特征点定位算法,该算法将人脸特征点定位分为关键点检测和标记两个独立的子问题。为了更好地在三维人脸上提取关键点,该算法提出了一个关键点检测方法:1)使用深度图和监督下降算法得到三维人脸特征点的粗略位置,提取特征点粗略位置的邻域作为关键点区域;2)提出了一种结合多个局部描述子的方法,对关键点区域内人脸点集的子集进行筛选,提取出关键点。在特征点标记阶段,使用关键点集生成候选特征点组合,选择与特征点模型匹配程度最高的组合,将组合中的候选点标记为特征点。基于FRGC v2.0和Bosphorus数据集对算法进行了实验评估,并与一些经典方法的结果进行了对比分析。FRGC v2.0库上的特征点的平均误差为2.85～3.81 mm,总体检测成功率为96.5%,其中中性、温和以及极端表情下检测成功率分别为97.5%、97.0%和93.3%。Bosphorus库上3种姿态下的检测成功率分别是92%、95%和88%。实验结果表明,该算法具有较好的精度和效率,对表情和小幅度的姿态变化具有较好的鲁棒性。
A coarse-to-fine algorithm for 3 D facial landmark localization based on keypoints is proposed. The landmark localization is divided into two independent subproblems, that are keypoint detection and labelling. To extract keypoints on 3 D faces more effectively, a keypoint detection method is proposed. First, coarse positions of landmarks are located by applying supervised descent method on depth images. The neighborhoods of landmarks′ coarse positions are extracted as keypoint regions. Second, a keypoint detection method is achieved by combining multiple local descriptors and filtering out the subset of the facial point set in keypoint regions. At the stage of labelling, a set of landmark candidates are generated from keypoints and those candidates best fitted the facial landmark model are labelled as the landmarks. The proposed algorithm is evaluated on FRGC v2.0 and Bosphorus datasets and compared with several state-of-the-art approaches. On the FRGC v2.0 dataset, the mean errors reach 2.85 mm to 3.81 mm for each landmark. The overall detection success rate is 96.5%, among which 97.5% for neutral expression, 97.0% for mild, 93.3% for extreme. On the Bosphorus dataset, the success rate reaches 92%, 95% and 88% respectively under three different poses. Experimental results show that the presented algorithm achieves good accuracy, efficiency and robustness against expression and small pose variation.
A coarse-to-fine algorithm for 3 D facial landmark localization based on keypoints is proposed. The landmark localization is divided into two independent subproblems, that are keypoint detection and labelling. To extract keypoints on 3 D faces more effectively, a keypoint detection method is proposed. First, coarse positions of landmarks are located by applying supervised descent method on depth images. The neighborhoods of landmarks′ coarse positions are extracted as keypoint regions. Second, a keypoint detection method is achieved by combining multiple local descriptors and filtering out the subset of the facial point set in keypoint regions. At the stage of labelling, a set of landmark candidates are generated from keypoints and those candidates best fitted the facial landmark model are labelled as the landmarks. The proposed algorithm is evaluated on FRGC v2.0 and Bosphorus datasets and compared with several state-of-the-art approaches. On the FRGC v2.0 dataset, the mean errors reach 2.85 mm to 3.81 mm for each landmark. The overall detection success rate is 96.5%, among which 97.5% for neutral expression, 97.0% for mild, 93.3% for extreme. On the Bosphorus dataset, the success rate reaches 92%, 95% and 88% respectively under three different poses. Experimental results show that the presented algorithm achieves good accuracy, efficiency and robustness against expression and small pose variation.
引文
[1] 汤兰兰, 盖绍彦, 达飞鹏,等. 基于网格纵横局部二值模式的三维人脸识别[J]. 仪器仪表学报, 2016, 37(6):1413-1420.TANG L L, GAI SH Y, DA F P, et al. A 3D face recognition method based on the local binary pattern from vertical and horizontal on the mesh[J]. Chinese Journal of Scientific Instrument, 2016, 37(6): 1413-1420.
[2] 马博宇,尉寅玮. 基于AdaBoost算法的人脸识别系统的研究与实现[J]. 仪器仪表学报, 2016, 37(增刊1): 162-167.MA B Y, WEI Y W. Design and implementation of face recognition system based on adaboost algorithm[J]. Chinese Journal of Scientific Instrument, 2016, 37(Suppl.1): 162-167.
[3] 陈志轩, 周大可, 黄经纬. 基于卷积神经网络的表情不变三维人脸识别[J]. 电子测量技术, 2017, 40(4): 157-161,171.CHEN ZH X, ZHOU D K, HUANG J W. Expression invariant 3D face recognition using convolutional neural networks[J]. Electronic Measurement Technology, 2017, 40(4): 157-161,171.
[4] 历艳琨, 毛建旭, 刘仁明. 基于特征点的3D人脸姿态跟踪[J]. 电子测量与仪器学报, 2016, 30(4): 605-612.LI Y K, MAO J X, LIU R M. 3D face pose tracking based on feature matching[J]. Journal of Electronic Measurement and Instrumentation, 2016, 30(4): 605-612.
[5] SANDBACH G, ZAFEIRIOU S, PANTIC M, et al. Static and dynamic 3D facial expression recognition: A comprehensive survey[J]. Image and Vision Computing, 2012, 30(10): 683-697.
[6] COLBRY D, STOCKMAN G, JAIN A. Detection of anchor points for 3D face verication[C]. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 2005: 118-118.
[7] FALTEMIER T C, BOWYER K W, FLYNN P J. Rotated profile signatures for robust 3D feature detection[C]. Proceedings of IEEE International Conference on Automatic Face & Gesture Recognition, 2008: 1-7.
[8] SEGUNDO M P, QUEIROLO C, BELLON O R P, et al. Automatic 3D facial segmentation and landmark detection[C]. Proceedings of International Conference on Image Analysis and Processing (ICIAP), 2007: 431-436.
[9] LU X G, JAIN A K. Multimodal facial feature extraction for automatic 3D face recognition[R]. Technical Report, 2005.
[10] NAIR P, CAVALLARO A. 3-D face detection, landmark localization, and registration using a point distribution model[J]. IEEE Transactions on Multimedia, 2009, 11(4): 611-623.
[11] PERAKIS P, PASSALIS G, THEOHARIS T, et al. 3D facial landmark detection under large yaw and expression variations[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(7): 1552-1564.
[12] SUKNO F M, WADDINGTON J L, WHELAN P F. 3-D facial landmark localization with asymmetry patterns and shape regression from incomplete local features[J]. IEEE Transactions on Cybernetics, 2015, 45(9): 1717-1730.
[13] ROSS A. Procrustes analysis[R]. Course Report, 2004.
[14] PERAKIS P, PASSALIS G, THEOHARIS T, et al. 3D facial landmark detection & face registration: A 3D facial landmark model & 3D local shape descriptors approach[R]. Technical Report, 2010.
[15] JOHNSON A E. Spin-images: A representation for 3-D surface matching[D]. Pittsburghers: Camegie Mellon University, 1997.
[16] CAMGZ N C, GKBERK B, AKARUN L. Facial landmark localizatin in depth images using supervised descent method[C]. Proceedings of IEEE Signal Processing and Communications Applications Conference, 2015: 1997-2000.
[17] XIONG X, DE LA TORRE F. Supervised descent method and its applications to face alignment[C]. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013: 532-539.
[18] KOENDERINK J J, VAN DOORN A J. Surface shape and curvature scales[J]. Image and Vision Computing, 1992, 10(8): 557-564.
[19] PHILLIPS P J, FLYNN P J, SCRUGGS T, et al. Overview of the face recognition grand challenge[C]. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005: 947-954.
[20] SAVRAN A, ALYüZ N, DIBEKLIO【math435z】, et al. Bosphorus database for 3D face analysis[C]. Proceedings of European Workshop on Biometrics and Identity Management, 2008: 47-56.
[21] GILANI S Z, SHAFAIT F, MIAN A. Shape-based automatic detection of a large number of 3D facial landmarks[C]. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2015: 4639-4648.
[22] PASSALIS G, PERAKIS P, THEOHARIS T, et al. Using facial symmetry to handle pose variations in real-world 3D face recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(10): 1938-1951.
[2] 马博宇,尉寅玮. 基于AdaBoost算法的人脸识别系统的研究与实现[J]. 仪器仪表学报, 2016, 37(增刊1): 162-167.MA B Y, WEI Y W. Design and implementation of face recognition system based on adaboost algorithm[J]. Chinese Journal of Scientific Instrument, 2016, 37(Suppl.1): 162-167.
[3] 陈志轩, 周大可, 黄经纬. 基于卷积神经网络的表情不变三维人脸识别[J]. 电子测量技术, 2017, 40(4): 157-161,171.CHEN ZH X, ZHOU D K, HUANG J W. Expression invariant 3D face recognition using convolutional neural networks[J]. Electronic Measurement Technology, 2017, 40(4): 157-161,171.
[4] 历艳琨, 毛建旭, 刘仁明. 基于特征点的3D人脸姿态跟踪[J]. 电子测量与仪器学报, 2016, 30(4): 605-612.LI Y K, MAO J X, LIU R M. 3D face pose tracking based on feature matching[J]. Journal of Electronic Measurement and Instrumentation, 2016, 30(4): 605-612.
[5] SANDBACH G, ZAFEIRIOU S, PANTIC M, et al. Static and dynamic 3D facial expression recognition: A comprehensive survey[J]. Image and Vision Computing, 2012, 30(10): 683-697.
[6] COLBRY D, STOCKMAN G, JAIN A. Detection of anchor points for 3D face verication[C]. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 2005: 118-118.
[7] FALTEMIER T C, BOWYER K W, FLYNN P J. Rotated profile signatures for robust 3D feature detection[C]. Proceedings of IEEE International Conference on Automatic Face & Gesture Recognition, 2008: 1-7.
[8] SEGUNDO M P, QUEIROLO C, BELLON O R P, et al. Automatic 3D facial segmentation and landmark detection[C]. Proceedings of International Conference on Image Analysis and Processing (ICIAP), 2007: 431-436.
[9] LU X G, JAIN A K. Multimodal facial feature extraction for automatic 3D face recognition[R]. Technical Report, 2005.
[10] NAIR P, CAVALLARO A. 3-D face detection, landmark localization, and registration using a point distribution model[J]. IEEE Transactions on Multimedia, 2009, 11(4): 611-623.
[11] PERAKIS P, PASSALIS G, THEOHARIS T, et al. 3D facial landmark detection under large yaw and expression variations[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(7): 1552-1564.
[12] SUKNO F M, WADDINGTON J L, WHELAN P F. 3-D facial landmark localization with asymmetry patterns and shape regression from incomplete local features[J]. IEEE Transactions on Cybernetics, 2015, 45(9): 1717-1730.
[13] ROSS A. Procrustes analysis[R]. Course Report, 2004.
[14] PERAKIS P, PASSALIS G, THEOHARIS T, et al. 3D facial landmark detection & face registration: A 3D facial landmark model & 3D local shape descriptors approach[R]. Technical Report, 2010.
[15] JOHNSON A E. Spin-images: A representation for 3-D surface matching[D]. Pittsburghers: Camegie Mellon University, 1997.
[16] CAMGZ N C, GKBERK B, AKARUN L. Facial landmark localizatin in depth images using supervised descent method[C]. Proceedings of IEEE Signal Processing and Communications Applications Conference, 2015: 1997-2000.
[17] XIONG X, DE LA TORRE F. Supervised descent method and its applications to face alignment[C]. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013: 532-539.
[18] KOENDERINK J J, VAN DOORN A J. Surface shape and curvature scales[J]. Image and Vision Computing, 1992, 10(8): 557-564.
[19] PHILLIPS P J, FLYNN P J, SCRUGGS T, et al. Overview of the face recognition grand challenge[C]. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005: 947-954.
[20] SAVRAN A, ALYüZ N, DIBEKLIO【math435z】, et al. Bosphorus database for 3D face analysis[C]. Proceedings of European Workshop on Biometrics and Identity Management, 2008: 47-56.
[21] GILANI S Z, SHAFAIT F, MIAN A. Shape-based automatic detection of a large number of 3D facial landmarks[C]. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2015: 4639-4648.
[22] PASSALIS G, PERAKIS P, THEOHARIS T, et al. Using facial symmetry to handle pose variations in real-world 3D face recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(10): 1938-1951.