基于融合约束局部模型的三维人脸特征点定位
|
摘要
提出基于特征融合约束局部模型的三维人脸特征点定位算法.该算法对每个特征点分别使用三维网格的深度信息和网格局部形状信息训练分类器,对分类器的响应进行融合.使用基于融合响应的正则化特征点均值漂移算法进行模型拟合,实现特征点定位.三维人脸特征点定位经常需要对每个特征点的候选点集进行遍历产生候选点组合,该算法使用模型拟合代替穷举搜索,避免了嵌套循环带来的快速增长的时间开销.使用FRGC v2.0和Bosphorus数据库,对算法进行实验评估. FRGC v2.0库上的特征点平均误差为2.48～4.12 mm,总体检测成功率为97.3%,其中中性、温和及极端表情下的检测成功率分别为97.6%、97.4%和95.5%. Bosphorus库上3种姿态下的检测成功率分别是94%、95%和89%.实验结果表明,提出方法具有较好的效果,对表情和小幅度的姿态变化具有较好的鲁棒性.
An algorithm for automatic detection of landmarks on three-dimensional faces was proposed by using a feature fusion based constrained local model. A classifier based on depth information and a classifier based on local shape information of three-dimensional meshes were trained for each landmark. The responses of two classifiers were merged and the regularized landmark mean shift algorithm was applied on fitting for the localization of landmarks. Traversing candidates of each landmark is usually necessary in three-dimensional facial landmark localization to generate candidate combinations. The problem of time overhead for nested loops that increases rapidly by using model fitting instead of exhaustive search was solved. The approach was evaluated based on threedimensional face databases: FRGC v2.0 and Bosphorus. The mean error of every landmark in the FRGC v2.0 is between 2.48 mm to 4.12 mm. The overall detection success rate is 97.3%, among which 97.6% for neutral expression, 97.4% for mild, 95.5% for extreme. On the Bosphorus database, the success rate of 94%, 95% and 89%was respectively achieved under three different poses. The experimental results show that the proposed approach is comparable to state-of-the-art methods in terms of its accuracy, and good robustness is achieved against expression and small pose variation.
An algorithm for automatic detection of landmarks on three-dimensional faces was proposed by using a feature fusion based constrained local model. A classifier based on depth information and a classifier based on local shape information of three-dimensional meshes were trained for each landmark. The responses of two classifiers were merged and the regularized landmark mean shift algorithm was applied on fitting for the localization of landmarks. Traversing candidates of each landmark is usually necessary in three-dimensional facial landmark localization to generate candidate combinations. The problem of time overhead for nested loops that increases rapidly by using model fitting instead of exhaustive search was solved. The approach was evaluated based on threedimensional face databases: FRGC v2.0 and Bosphorus. The mean error of every landmark in the FRGC v2.0 is between 2.48 mm to 4.12 mm. The overall detection success rate is 97.3%, among which 97.6% for neutral expression, 97.4% for mild, 95.5% for extreme. On the Bosphorus database, the success rate of 94%, 95% and 89%was respectively achieved under three different poses. The experimental results show that the proposed approach is comparable to state-of-the-art methods in terms of its accuracy, and good robustness is achieved against expression and small pose variation.
引文
[1]郭梦丽,达飞鹏,邓星,等.基于关键点和局部特征的三维人脸识别[J].浙江大学学报:工学版,2017,51(3):584-589.GUO Meng-li,DA Fei-peng,DENG Xing,et al.3Dface recognition based on keypoints and local feature[J].Journal of Zhejiang University:Engineering Science,2017,51(3):584-589.
[2]汤兰兰,盖绍彦,达飞鹏,等.基于网格纵横局部二值模式的三维人脸识别[J].仪器仪表学报,2016,37(6):1413-1420.TANG Lan-lan,GAI Shao-yan,DA Fei-peng,et al.A3D face recognition method based on the local binary pattern from vertical and horizontal on the mesh[J].Chinese Journal of Scientific Instrument,2016,37(6):1413-1420.
[3]DENG X,DA F,SHAO H.Efficient 3D face recognition using local covariance descriptor and Riemannian kernel sparse coding[J].Computers and Electrical Engineering,2017,62(8):81-91.
[4]DERKACH D,SUKNO F M.Local shape spectrum analysis for 3D facial expression recognition[C]//Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition(FG).Washington DC:IEEE,2017:41-47.
[5]ZHAO Q,OKADA K,ROSENBAUM K,et al.Digital facial dysmorphology for genetic screening:Hierarchical constrained local model using ICA[J].Medical Image Analysis,2014,18(5):699-710.
[6]COOTES T F,TAYLOR C J,COOPER D H,et al.Active shape models-their training and application[J].Computer Vision and Image Understanding,1995,61(1):38-59.
[7]EDWARDS G J,COOTES T F,TAYLOR C J.Face recognition using active appearance models[C]//Proceedings of European Conference on Computer Vision(ECCV).Berlin:Springer,1998:581-595.
[8]CRISTINACCE D,COOTES T.Automatic feature localisation with constrained local models[J].Pattern Recognition,2008,41(10):3054-3067.
[9]BALTRU?AITIS T,ROBINSON P,MORENCY L P.3D constrained local model for rigid and non-rigid facial tracking[C]//Proceedings of IEEEConference on Computer Vision and Pattern Recognition(CVPR).Rhode Island:IEEE,2012:2610-2617.
[10]SARAGIH J M,LUCEY S,COHN J F.Deformable model fitting by regularized landmark meanshift[J].International Journal of Computer Vision,2011,91(2):200-215.
[11]CHENG S,ZAFEIRIOU S,ASTHANA A,et al.3Dfacial geometric features for constrained local model[C]//Proceedings of IEEE International Conference on Image Processing(ICIP).Paris:IEEE,2014:1425-1429.
[12]NAIR P,CAVALLARO A.3-D face detection,landmark localization,and registration using a point distribution model[J].IEEE Transactions on Multimedia,2009,11(4):611-623.
[13]PERAKIS P,PASSALIS G,THEOHARIS T,et al.3D facial landmark detection under large yaw and expression variations[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(7):1552-1564.
[14]CREUSOT C,PEARS N,AUSTIN J.A machinelearning approach to keypoint detection and landmarking on 3D meshes[J].International Journal of Computer Vision,2013,102(1-3):146-179.
[15]SUKNO F M,WADDINGTON J L,WHELAN P F.3-D facial landmark localization with asymmetry patterns and shape regression from incomplete local features[J].IEEE Transactions on Cybernetics,2015,45(9):1717-1730.
[16]GOWER J C.Generalized procrustes analysis[J].Psychometrika,1975,40(1):33-51.
[17]MOGHADDAM B,PENTLAND A.Probabilistic visual learning for object representation[J].IEEETransactions on Pattern Analysis and Machine Intelligence,1997,19(7):696-710.
[18]JOHNSON A E.Spin-images:a representation for3D surface matching[D].Pittsburgh:Carnegie Mellon University,1997.
[19]PHILLIPS P J,FLYNN P J,SCRUGGS T,et al.Overview of the face recognition grand challenge[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR).San Diego:IEEE,2005:947-954.
[20]SAVRAN A,ALYüZ N,DIBEKLIO?LU H,et al.Bosphorus database for 3D face analysis[C]//Proceedings of European Workshop on Biometrics and Identity Management(BIOID).Berlin:Springer,2008:47-56.
[21]CAMG?Z N C,G?KBERK B,AKARUN L.Facial landmark localization in depth images using supervised descent method[C]//Proceedings of Signal Processing and Communications Applications Conference(SIU).Malatya:IEEE,2015:1997-2000.
[2]汤兰兰,盖绍彦,达飞鹏,等.基于网格纵横局部二值模式的三维人脸识别[J].仪器仪表学报,2016,37(6):1413-1420.TANG Lan-lan,GAI Shao-yan,DA Fei-peng,et al.A3D face recognition method based on the local binary pattern from vertical and horizontal on the mesh[J].Chinese Journal of Scientific Instrument,2016,37(6):1413-1420.
[3]DENG X,DA F,SHAO H.Efficient 3D face recognition using local covariance descriptor and Riemannian kernel sparse coding[J].Computers and Electrical Engineering,2017,62(8):81-91.
[4]DERKACH D,SUKNO F M.Local shape spectrum analysis for 3D facial expression recognition[C]//Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition(FG).Washington DC:IEEE,2017:41-47.
[5]ZHAO Q,OKADA K,ROSENBAUM K,et al.Digital facial dysmorphology for genetic screening:Hierarchical constrained local model using ICA[J].Medical Image Analysis,2014,18(5):699-710.
[6]COOTES T F,TAYLOR C J,COOPER D H,et al.Active shape models-their training and application[J].Computer Vision and Image Understanding,1995,61(1):38-59.
[7]EDWARDS G J,COOTES T F,TAYLOR C J.Face recognition using active appearance models[C]//Proceedings of European Conference on Computer Vision(ECCV).Berlin:Springer,1998:581-595.
[8]CRISTINACCE D,COOTES T.Automatic feature localisation with constrained local models[J].Pattern Recognition,2008,41(10):3054-3067.
[9]BALTRU?AITIS T,ROBINSON P,MORENCY L P.3D constrained local model for rigid and non-rigid facial tracking[C]//Proceedings of IEEEConference on Computer Vision and Pattern Recognition(CVPR).Rhode Island:IEEE,2012:2610-2617.
[10]SARAGIH J M,LUCEY S,COHN J F.Deformable model fitting by regularized landmark meanshift[J].International Journal of Computer Vision,2011,91(2):200-215.
[11]CHENG S,ZAFEIRIOU S,ASTHANA A,et al.3Dfacial geometric features for constrained local model[C]//Proceedings of IEEE International Conference on Image Processing(ICIP).Paris:IEEE,2014:1425-1429.
[12]NAIR P,CAVALLARO A.3-D face detection,landmark localization,and registration using a point distribution model[J].IEEE Transactions on Multimedia,2009,11(4):611-623.
[13]PERAKIS P,PASSALIS G,THEOHARIS T,et al.3D facial landmark detection under large yaw and expression variations[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(7):1552-1564.
[14]CREUSOT C,PEARS N,AUSTIN J.A machinelearning approach to keypoint detection and landmarking on 3D meshes[J].International Journal of Computer Vision,2013,102(1-3):146-179.
[15]SUKNO F M,WADDINGTON J L,WHELAN P F.3-D facial landmark localization with asymmetry patterns and shape regression from incomplete local features[J].IEEE Transactions on Cybernetics,2015,45(9):1717-1730.
[16]GOWER J C.Generalized procrustes analysis[J].Psychometrika,1975,40(1):33-51.
[17]MOGHADDAM B,PENTLAND A.Probabilistic visual learning for object representation[J].IEEETransactions on Pattern Analysis and Machine Intelligence,1997,19(7):696-710.
[18]JOHNSON A E.Spin-images:a representation for3D surface matching[D].Pittsburgh:Carnegie Mellon University,1997.
[19]PHILLIPS P J,FLYNN P J,SCRUGGS T,et al.Overview of the face recognition grand challenge[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR).San Diego:IEEE,2005:947-954.
[20]SAVRAN A,ALYüZ N,DIBEKLIO?LU H,et al.Bosphorus database for 3D face analysis[C]//Proceedings of European Workshop on Biometrics and Identity Management(BIOID).Berlin:Springer,2008:47-56.
[21]CAMG?Z N C,G?KBERK B,AKARUN L.Facial landmark localization in depth images using supervised descent method[C]//Proceedings of Signal Processing and Communications Applications Conference(SIU).Malatya:IEEE,2015:1997-2000.