实时三维人脸特征点定位
|
摘要
为了能够自动、实时地标定三维人脸曲面上主要特征点的位置,如鼻尖点和内眼点等,提出一种实时定位三维人脸特征点的算法.首先设计了一种基于曲面划分、直方图统计的三维局部曲面描述子——球划分直方图描述子,并使用该描述子逐点提取三维曲面顶点的局部曲面信息;其次计算其相似度,以定位三维人脸曲面上的主要特征点;在采用GPU并行运算后,能够有效地对三维人脸特征点进行实时定位.在三维人脸数据库FRGC v1.0以及BU-3DFE中的实验结果表明,与其他算法相比,该算法在三维人脸曲面上具有较强的三维人脸特征点定位能力.
In order to automatically find facial landmarks of a 3Dfacial shape,such as nose-tip and inner eye corners,this paper presents an effective approach to automatically detect landmarks of a 3D face in real time.A local surface descriptor based on partition and histogram,named sphere partition histogram descriptor,is designed to characterize local surface at a point on 3Dface shape.Then we exploit the SPIDER descriptor to localize facial landmarks.The proposed algorithm is suitable for acceleration by GPU parallel computing.The experiments on FRGC v1.0and BU-3DFE show that our algorithm outperforms other existing methods on 3Dfacial landmarks localization.
In order to automatically find facial landmarks of a 3Dfacial shape,such as nose-tip and inner eye corners,this paper presents an effective approach to automatically detect landmarks of a 3D face in real time.A local surface descriptor based on partition and histogram,named sphere partition histogram descriptor,is designed to characterize local surface at a point on 3Dface shape.Then we exploit the SPIDER descriptor to localize facial landmarks.The proposed algorithm is suitable for acceleration by GPU parallel computing.The experiments on FRGC v1.0and BU-3DFE show that our algorithm outperforms other existing methods on 3Dfacial landmarks localization.
引文
[1]Johnson A E,Hebert M.Using spin images for efficientobject recognition in cluttered 3D scenes[J].IEEETransactions on Pattern Analysis and Machine Intelligence,1999,21(5):433-449
[2]Bowyer K W,Chang K,Flynn P.A survey of approaches andchallenges in 3Dand multi-modal 3D+2Dface recognition[J].Computer Vision and Image Understanding,2006,101(1):1-15
[3]Phillips P J,Flynn P J,Scruggs T,et al.Overview of theface recognition grand challenge[C]//Proceedings of IEEEComputer Society Conference on Vision and PatternRecognition.Los Alamitos:IEEE Computer Society Press,2005,1:947-954
[4]Wang Y M,Tang X O,Liu J Z,et al.3Dface recognition bylocal shape difference boosting[M]//Lecture Notes inComputer Science.Heidelberg:Springer,2008,5302:603-616
[5]Pan G,Wu Z H,Pan Y H.Automatic 3Dface verificationfrom range data[C]//Proceedings of IEEE InternationalConference on Acoustics,Speech,and Signal Processing.LosAlamitos:IEEE Computer Society Press,2003,3:193-196
[6]Pan G,Han S,Wu Z,et al.3Dface recognition usingmapped depth images[C]//Proceedings of IEEE ComputerSociety Conference on Computer Vision and PatternRecognition-Workshops.Los Alamitos:IEEE ComputerSociety Press,2005:175
[7]Wang Y M,Pan G,Wu Z H.3Dface recognition in thepresence of expression:a guidance-based constraintdeformation approach[C]//Proceedings of IEEE Conferenceon Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2007:1-7
[8]Pan G,Han S,Wu Z H,et al.Removal of 3D facialexpressions:a learning-based approach[C]//Proceedings ofIEEE Conference on Computer Vision and PatternRecognition.Los Alamitos:IEEE Computer Society Press,2010:2614-2621
[9]Pan G,Han S,Wu Z H.Hallucinating 3Dfacial shapes[C]//Proceedings of IEEE Conference on Computer Visionand Pattern Recognition.Los Alamitos:IEEE ComputerSociety Press,2008:1-8
[10]Wang J,Yin L J,Wei X Z,et al.3Dfacial expressionrecognition based on primitive surface feature distribution[C]//Proceedings of IEEE Computer Society Conference onComputer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2006,2:1399-1406
[11]Han Song,Pan Gang,Fu Junkang,et al.GPU-acceleratedparallel realistic 3Dfacial expression synthesis[J].Journal ofComputer-Aided Design&Computer Graphics,2011,23(5):747-755(in Chinese)(韩松,潘纲,傅俊康,等.GPU加速的三维人脸真实表情合成[J].计算机辅助设计与图形学学报,2011,23(5):747-755)
[12]Blanz V,Vetter T.A morphable model for the synthesis of3Dfaces[C]//Proceedings of the 26th Annual Conference onComputer Graphics and Interactive Techniques.New York:ACM Press,1999:187-194
[13]Huang Xiaoqin,Lin Yuxu,Song Mingli,et al.3Dfacialexpression synthesis based on nolinear co-learning[J].Journal of Computer-Aided Design&Computer Graphics,2011,23(2):363-370(in Chinese)(黄晓钦,林裕旭,宋明黎,等.非线性联合学习的三维人脸表情合成方法[J].计算机辅助设计与图形学学报,2011,23(2):363-370)
[14]Colbry D,Stockman G,Jain A.Detection of anchor pointsfor 3Dface verification[C]//Proceedings of IEEE ComputerSociety Conference on Computer Vision and PatternRecognition-Workshops.Los Alamitos:IEEE ComputerSociety Press,2005:118
[15]Han Song,Pan Gang,Wang Yueming,et al.3Dnose:anovel biometrics[J].Journal of Computer-Aided Design&Computer Graphics,2008,20(1):38-42(in Chinese)(韩松,潘纲,王跃明,等.三维鼻形:一种新的生物特征识别模式[J].计算机辅助设计与图形学学报,2008,20(1):38-42)
[16]Yin L J,Wei X Z,Sun Y,et al.A 3Dfacial expressiondatabase for facial behavior research[C]//Proceedings of the7th International Conference on Automatic Face and GestureRecognition.Los Alamitos:IEEE Computer Society Press,2006:211-216
[17]Lee Y,Yi T.3Dface recognition using multiple features forlocal depth information[C]//Proceedings of the 4th EurasipConference Focused on Video/Image Processing andMultimedia Communications.Los Alamitos:IEEE ComputerSociety Press,2003,1:429-434
[18]Hesher C,Srivastava A,Erlebacher G.A novel technique forface recognition using range imaging[C]//Proceedings of the7th International Symposium on Signal Processing and ItsApplications.Los Alamitos:IEEE Computer Society Press,2003,2:201-204
[19]Malassiotis S,Strintzis M G.Robust real-time 3Dhead poseestimation from range data[J].Pattern Recognition,2005,38(8):1153-1165
[20]Beumier C,Acheroy M.Automatic 3Dface authentication[J].Image and Vision Computing,2000,18(4):315-321
[21]Wu Y,Pan G,Wu Z H.Face authentication based onmultiple profiles extracted from range data[M]//LectureNotes in Computer Science.Heidelberg:Springer,2003,2688:515-522
[22]Gordon G G.Face recognition based on depth and curvaturefeatures[C]//Proceedings of IEEE Computer SocietyConference on Computer Vision and Pattern Recognition.LosAlamitos:IEEE Computer Society Press,1992:808-810
[23]Nair P,Cavallaro A.Region segmentation and feature pointextraction on 3Dfaces using apoint distribution model[C]//Proceedings of IEEE International Conference on ImageProcessing.Los Alamitos:IEEE Computer Society Press,2007,3:85-88
[24]Akagunduz E,Ulusoy I.3D object representation usingtransform and scale invariant 3Dfeatures[C]//Proceedings ofthe 11th International Conference on Computer Vision.LosAlamitos:IEEE Computer Society Press,2007,1:1-8
[25]Chang K I,Bowyer W,Flynn P J.Multiple nose regionmatching for 3D face recognition under varying facialexpression[J].IEEE Transactions on Pattern Analysis andMachine Intelligence,2006,28(10):1695-1700
[26]Xu C H,Tan T N,Wang Y H,et al.Combining localfeatures for robust nose location in 3Dfacial data[J].PatternRecognition Letters,2006,27(13):1487-1494
[27]Faltemier T C,Bowyer K W,Flynn P J.Rotated profilesignatures for robust 3Dfeature detection[C]//Proceedings ofthe 8th IEEE International Conference on Automatic Face&Gesture Recognition.Los Alamitos:IEEE Computer SocietyPress,2008:1-7
[28]Breitenstein M D,Kuettel D,Weise T,et al.Real-time facepose estimation from single range images[C]//Proceedings ofIEEE Conference on Computer Vision and PatternRecognition.Los Alamitos:IEEE Computer Society Press,2008:1-8
[29]Fanelli G,Gall J,Van Gool L.Real time head poseestimation with random regression forests[C]//Proceedingsof IEEE Conference on IEEE Conference on Computer Visionand Pattern Recognition.Los Alamitos:IEEE ComputerSociety Press,2011:617-624
[30]Boehnen C,Russ T.A fast multi-modal approach to facialfeature detection[C]//Proceedings of the 7th IEEEWorkshops on Application of Computer Vision.LosAlamitos:IEEE Computer Society Press,2005,1:135-142
[31]Xiao J,Baker S,Matthews I,et al.Real-time combined 2D+3Dactive appearance models[C]//Proceedings of IEEEComputer Society Conference on Computer Vision and PatternRecognition.Los Alamitos:IEEE Computer Society Press,2004:535-542
[32]Wang Y J,Chua C S,Ho Y K.Facial feature detection andface recognition from 2D and 3D images[J].PatternRecognition Letters,2002,23(10):1191-1202
[33]Horn B K P.Extended gaussian image[J].Proceedings ofthe IEEE,1984,72(12):1671-1686
[34]Wu Enhua,Liu Youquan.General purpose computation onGPU[J].Journal of Computer-Aided Design&ComputerGraphics,2004,16(5):601-602(in Chinese)(吴恩华,柳有权.基于图形处理器(GPU)的通用计算[J].计算机辅助设计与图形学学报,2004,16(5):601-602)
[35]Cootes T F,Taylor C J,Cooper D H,et al.Active shapemodels-their training and application[J].Computer Visionand Image Understanding,1995,61(1):38-59
[36]Garland M,Heckbert P S.Surface simplification usingquadric error metrics[C]//Proceedings of the 24th AnnualConference on Computer Graphics and Interactive Techniques(SIGGRAPH'97).New York:ACM Press,1997:209-216
[37]Osada R,Funkhouser T,Chazelle B,et al.Shapedistributions[J].ACM Transactions on Graphics,2002,21(4):807-832
[38]Ankerst M,Kastenmuller G,Kriegel H P,et al.3Dshapehistograms for similarity search and classification in spatialdatabases[M]//Lecture Notes in Computer Science.Heidelberg:Springer,1999,1651:207-226
[2]Bowyer K W,Chang K,Flynn P.A survey of approaches andchallenges in 3Dand multi-modal 3D+2Dface recognition[J].Computer Vision and Image Understanding,2006,101(1):1-15
[3]Phillips P J,Flynn P J,Scruggs T,et al.Overview of theface recognition grand challenge[C]//Proceedings of IEEEComputer Society Conference on Vision and PatternRecognition.Los Alamitos:IEEE Computer Society Press,2005,1:947-954
[4]Wang Y M,Tang X O,Liu J Z,et al.3Dface recognition bylocal shape difference boosting[M]//Lecture Notes inComputer Science.Heidelberg:Springer,2008,5302:603-616
[5]Pan G,Wu Z H,Pan Y H.Automatic 3Dface verificationfrom range data[C]//Proceedings of IEEE InternationalConference on Acoustics,Speech,and Signal Processing.LosAlamitos:IEEE Computer Society Press,2003,3:193-196
[6]Pan G,Han S,Wu Z,et al.3Dface recognition usingmapped depth images[C]//Proceedings of IEEE ComputerSociety Conference on Computer Vision and PatternRecognition-Workshops.Los Alamitos:IEEE ComputerSociety Press,2005:175
[7]Wang Y M,Pan G,Wu Z H.3Dface recognition in thepresence of expression:a guidance-based constraintdeformation approach[C]//Proceedings of IEEE Conferenceon Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2007:1-7
[8]Pan G,Han S,Wu Z H,et al.Removal of 3D facialexpressions:a learning-based approach[C]//Proceedings ofIEEE Conference on Computer Vision and PatternRecognition.Los Alamitos:IEEE Computer Society Press,2010:2614-2621
[9]Pan G,Han S,Wu Z H.Hallucinating 3Dfacial shapes[C]//Proceedings of IEEE Conference on Computer Visionand Pattern Recognition.Los Alamitos:IEEE ComputerSociety Press,2008:1-8
[10]Wang J,Yin L J,Wei X Z,et al.3Dfacial expressionrecognition based on primitive surface feature distribution[C]//Proceedings of IEEE Computer Society Conference onComputer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2006,2:1399-1406
[11]Han Song,Pan Gang,Fu Junkang,et al.GPU-acceleratedparallel realistic 3Dfacial expression synthesis[J].Journal ofComputer-Aided Design&Computer Graphics,2011,23(5):747-755(in Chinese)(韩松,潘纲,傅俊康,等.GPU加速的三维人脸真实表情合成[J].计算机辅助设计与图形学学报,2011,23(5):747-755)
[12]Blanz V,Vetter T.A morphable model for the synthesis of3Dfaces[C]//Proceedings of the 26th Annual Conference onComputer Graphics and Interactive Techniques.New York:ACM Press,1999:187-194
[13]Huang Xiaoqin,Lin Yuxu,Song Mingli,et al.3Dfacialexpression synthesis based on nolinear co-learning[J].Journal of Computer-Aided Design&Computer Graphics,2011,23(2):363-370(in Chinese)(黄晓钦,林裕旭,宋明黎,等.非线性联合学习的三维人脸表情合成方法[J].计算机辅助设计与图形学学报,2011,23(2):363-370)
[14]Colbry D,Stockman G,Jain A.Detection of anchor pointsfor 3Dface verification[C]//Proceedings of IEEE ComputerSociety Conference on Computer Vision and PatternRecognition-Workshops.Los Alamitos:IEEE ComputerSociety Press,2005:118
[15]Han Song,Pan Gang,Wang Yueming,et al.3Dnose:anovel biometrics[J].Journal of Computer-Aided Design&Computer Graphics,2008,20(1):38-42(in Chinese)(韩松,潘纲,王跃明,等.三维鼻形:一种新的生物特征识别模式[J].计算机辅助设计与图形学学报,2008,20(1):38-42)
[16]Yin L J,Wei X Z,Sun Y,et al.A 3Dfacial expressiondatabase for facial behavior research[C]//Proceedings of the7th International Conference on Automatic Face and GestureRecognition.Los Alamitos:IEEE Computer Society Press,2006:211-216
[17]Lee Y,Yi T.3Dface recognition using multiple features forlocal depth information[C]//Proceedings of the 4th EurasipConference Focused on Video/Image Processing andMultimedia Communications.Los Alamitos:IEEE ComputerSociety Press,2003,1:429-434
[18]Hesher C,Srivastava A,Erlebacher G.A novel technique forface recognition using range imaging[C]//Proceedings of the7th International Symposium on Signal Processing and ItsApplications.Los Alamitos:IEEE Computer Society Press,2003,2:201-204
[19]Malassiotis S,Strintzis M G.Robust real-time 3Dhead poseestimation from range data[J].Pattern Recognition,2005,38(8):1153-1165
[20]Beumier C,Acheroy M.Automatic 3Dface authentication[J].Image and Vision Computing,2000,18(4):315-321
[21]Wu Y,Pan G,Wu Z H.Face authentication based onmultiple profiles extracted from range data[M]//LectureNotes in Computer Science.Heidelberg:Springer,2003,2688:515-522
[22]Gordon G G.Face recognition based on depth and curvaturefeatures[C]//Proceedings of IEEE Computer SocietyConference on Computer Vision and Pattern Recognition.LosAlamitos:IEEE Computer Society Press,1992:808-810
[23]Nair P,Cavallaro A.Region segmentation and feature pointextraction on 3Dfaces using apoint distribution model[C]//Proceedings of IEEE International Conference on ImageProcessing.Los Alamitos:IEEE Computer Society Press,2007,3:85-88
[24]Akagunduz E,Ulusoy I.3D object representation usingtransform and scale invariant 3Dfeatures[C]//Proceedings ofthe 11th International Conference on Computer Vision.LosAlamitos:IEEE Computer Society Press,2007,1:1-8
[25]Chang K I,Bowyer W,Flynn P J.Multiple nose regionmatching for 3D face recognition under varying facialexpression[J].IEEE Transactions on Pattern Analysis andMachine Intelligence,2006,28(10):1695-1700
[26]Xu C H,Tan T N,Wang Y H,et al.Combining localfeatures for robust nose location in 3Dfacial data[J].PatternRecognition Letters,2006,27(13):1487-1494
[27]Faltemier T C,Bowyer K W,Flynn P J.Rotated profilesignatures for robust 3Dfeature detection[C]//Proceedings ofthe 8th IEEE International Conference on Automatic Face&Gesture Recognition.Los Alamitos:IEEE Computer SocietyPress,2008:1-7
[28]Breitenstein M D,Kuettel D,Weise T,et al.Real-time facepose estimation from single range images[C]//Proceedings ofIEEE Conference on Computer Vision and PatternRecognition.Los Alamitos:IEEE Computer Society Press,2008:1-8
[29]Fanelli G,Gall J,Van Gool L.Real time head poseestimation with random regression forests[C]//Proceedingsof IEEE Conference on IEEE Conference on Computer Visionand Pattern Recognition.Los Alamitos:IEEE ComputerSociety Press,2011:617-624
[30]Boehnen C,Russ T.A fast multi-modal approach to facialfeature detection[C]//Proceedings of the 7th IEEEWorkshops on Application of Computer Vision.LosAlamitos:IEEE Computer Society Press,2005,1:135-142
[31]Xiao J,Baker S,Matthews I,et al.Real-time combined 2D+3Dactive appearance models[C]//Proceedings of IEEEComputer Society Conference on Computer Vision and PatternRecognition.Los Alamitos:IEEE Computer Society Press,2004:535-542
[32]Wang Y J,Chua C S,Ho Y K.Facial feature detection andface recognition from 2D and 3D images[J].PatternRecognition Letters,2002,23(10):1191-1202
[33]Horn B K P.Extended gaussian image[J].Proceedings ofthe IEEE,1984,72(12):1671-1686
[34]Wu Enhua,Liu Youquan.General purpose computation onGPU[J].Journal of Computer-Aided Design&ComputerGraphics,2004,16(5):601-602(in Chinese)(吴恩华,柳有权.基于图形处理器(GPU)的通用计算[J].计算机辅助设计与图形学学报,2004,16(5):601-602)
[35]Cootes T F,Taylor C J,Cooper D H,et al.Active shapemodels-their training and application[J].Computer Visionand Image Understanding,1995,61(1):38-59
[36]Garland M,Heckbert P S.Surface simplification usingquadric error metrics[C]//Proceedings of the 24th AnnualConference on Computer Graphics and Interactive Techniques(SIGGRAPH'97).New York:ACM Press,1997:209-216
[37]Osada R,Funkhouser T,Chazelle B,et al.Shapedistributions[J].ACM Transactions on Graphics,2002,21(4):807-832
[38]Ankerst M,Kastenmuller G,Kriegel H P,et al.3Dshapehistograms for similarity search and classification in spatialdatabases[M]//Lecture Notes in Computer Science.Heidelberg:Springer,1999,1651:207-226