变形图驱动的人体与脸部网格模板拟合系统
|
摘要
为了建立统一的人体模型数据库/人脸模型数据库,实现了一个用标准三维人体/人脸网格模型拟合扫描数据的系统.该系统为人体模型构建特征点候选区域,用马尔可夫网络对候选区域进行概率预测得到人体特征点的位置;对于人脸模型,采用基于回归树的算法在多幅不同角度的人脸图像上分别检测其二维特征点,融合得到准确的三维特征点位置;得到特征点后,使用相似变换对标准模板和扫描模型的位置朝向以及尺度进行统一;通过特征点引导的变形图算法对标准模板和扫描模型的形状和姿态进行粗配准;最后使用基于稠密点对应的顶点仿射变换拟合得到变形后的标准模板.对CAESAR数据集中的人体模型以及扫描得到的人体和人脸模型均进行了拟合,实验结果表明,该系统能够精确地捕捉扫描数据的几何形状.
To build a unified human body model database/face model database, we propose an efficient system using a standard 3 D human body mesh/face mesh model to fit the scanning data, so that the different shape models in the database have the same connectivity. It constructs a candidate area for detecting landmarks, and then uses a Markov network to detect the landmarks. As for human face, it employs a regression tree based algorithm detect 2 D landmarks in face images captured from different viewing angles. These 2 D landmarks are then fused to compute accurate 3 D facial landmarks. After that, a similarity transformation is performed to align the standard template and the scanning model. Then the shape and pose of the standard template and the scanning model are roughly registered by using the deformable graph algorithm guided under feature points. Finally, the deformed standard template is fitted by vertex affine transformation based on dense point correspondence. The system is used to fit the human body models in the CAESAR dataset as well as the raw human body and face models. Experimental results show that the system can capture the geometry of the given data accurately.
To build a unified human body model database/face model database, we propose an efficient system using a standard 3 D human body mesh/face mesh model to fit the scanning data, so that the different shape models in the database have the same connectivity. It constructs a candidate area for detecting landmarks, and then uses a Markov network to detect the landmarks. As for human face, it employs a regression tree based algorithm detect 2 D landmarks in face images captured from different viewing angles. These 2 D landmarks are then fused to compute accurate 3 D facial landmarks. After that, a similarity transformation is performed to align the standard template and the scanning model. Then the shape and pose of the standard template and the scanning model are roughly registered by using the deformable graph algorithm guided under feature points. Finally, the deformed standard template is fitted by vertex affine transformation based on dense point correspondence. The system is used to fit the human body models in the CAESAR dataset as well as the raw human body and face models. Experimental results show that the system can capture the geometry of the given data accurately.
引文
[1]Sederberg T W,Parry S R.Free-form deformation of solid geometric models[C]//Proceedings of the 13th Annual Conference on Computer Graphics and Interactive Techniques.New York:ACM Press,1986:151-160
[2]Kobbelt L,Campagna S,Vorsatz J,et al.Interactive multiresolution modeling on arbitrary meshes[C]//Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques.New York:ACM Press,1998:105-114
[3]Botsch M,Kobbelt L.An intuitive framework for real-time freeform modeling[J].ACM Transactions on Graphics,2004,23(3):630-634
[4]Allen B,Curless B,Popovic Z.The space of human body shapes:reconstruction and parameterization from range scans[J].ACM Transaction on Graphics,2003,22(3):587-594
[5]Sumner R W,Schmid J,Pauly M.Embedded deformation for shape manipulation[J].ACM Transactions on Graphics,2007,26(3):Article No.80
[6]Goodall C.Procrustes methods in the statistical analysis of shape[J].Journal of the Royal Statistical Society,1991,53(2):285-339
[7]Magnenat-Thalmann N,Laperrire R,Thalmann D.Joint-dependent local deformations for hand animation and object grasping[C]//Proceedings of on Graphics Interface.Toronto:Canadian Information Processing Society Press,1988:26-33
[8]Anguelov D,Srinivasan P,Koller D,et al.SCAPE:shape completion and animation of people[J].ACM Transactions on Graphics,2005,24(3):408-416
[9]Loper M,Mahmood N,Romero J,et al.SMPL:a skinned multi-person linear model[J].ACM Transactions on Graphics,2015,34(6):Article No.248
[10]Li Guiqing,Ma Weiyin,Bao Hujun.Fitting system using loop subdivision surfaces with sharp features[J].Journal of Computer-Aided Design&Computer Graphics,2005,17(6):1179-1185(in Chinese)(李桂清,马维银,鲍虎军.带尖锐特征的Loop细分曲面拟合系统[J].计算机辅助设计与图形学学报,2005,17(6):1179-1185)
[11]Wang Y P,Li G Q,Zeng Z C,et al.Articulated-motion-aware sparse localized decomposition[J].Computer Graphics Forum,2017,36(8):247-259
[12]Hirshberg D A,Loper M,Rachlin E,et al.Coregistration:simultaneous alignment and modeling of articulated 3Dshape[C]//Proceedings of the 12th European Conference on Computer Vision.Heidelherg:Springer,2012:242-255
[13]Zhou Jin,Pan Jianjiang,Tong Jing,et al.Fast 3D human body reconstruction from Kinects[J].Journal of Computer-Aided Design&Computer Graphics,2013,25(6):873-879(in Chinese)(周瑾,潘建江,童晶,等.使用Kinect快速重建三维人体[J].计算机辅助设计与图形学学报,2013,25(6):873-879)
[14]Dong Hongwei.Mesh deformation based 3D facial modeling from images[J].Journal of Computer-Aided Design&Computer Graphics,2012,24(7):932-940(in Chinese)(董洪伟.基于网格变形的从图像重建三维人脸[J].计算机辅助设计与图形学学报,2012,24(7):932-940)
[15]Gu L,Kanade T.3D alignment of face in a single image[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2006:1305-1312
[16]Gou C,Wu Y,Wang F Y,et al.Shape augmented regression for 3D face alignment[C]//Proceedings of European Conference on Computer Vision.Heidelherg:Springer,2016:604-615
[17]Azouz Z B,Shu C,Mantel A.Automatic locating of anthropometric landmarks on 3D human models[C]//Proceedings of the 3rd International Symposium on 3D Data Processing,Visualization,and Transmission.Los Alamitos:IEEE Computer Society Press,2006:750-757
[18]Johnson A E.Spin-Images:a representation for 3-D surface matching[D].Pittsburgh:Carnegie Mellon University,1997
[19]Wuhrer S,Azouz Z B,Shu C.Semi-automatic prediction of landmarks on human models in varying poses[C]//Proceedings of the Canadian Conference on Computer and Robot Vision.Los Alamitos:IEEE Computer Society Press,2010:136-142
[20]Li C Y.Spectral geometric methods for deformable 3D shape retrieval[D].Montreal:Concordia University,2013
[21]Han H,Nam Y.Automatic body landmark identification for various body figures[J].International Journal of Industrial Ergonomics,2011,41(6):592-606
[22]Li Yuelong,Jin Yan,Wang Jianming.Face feature points extraction:a review[J].Chinese Journal of Computers,2016,39(7):1356-1374(in Chinese)(李月龙,靳彦,汪剑鸣.人脸特征点提取方法综述[J].计算机学报,2016,39(7):1356-1374)
[23]Cootes T F,Taylor C J.Statistical models of appearance for computer vision[R].Manchester:University of Manchester,Technical Report,2004
[24]Tong Y,Wang Y,Zhu Z W,et al.Robust facial feature tracking under varying face pose and facial expression[J].Pattern Recognition,2007,40(11):3195-3208
[25]Kazemi V,Sullivan J.One millisecond face alignment with an ensemble of regression trees[C]//Proceedings of the IEEEConference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2014:1867-1874
[26]Wang Migong,Chen Duansheng,Lin Chao.Automatic location of 3D facial feature points based on local shape map[J].Journal of Computer Applications,2010,30(5):1255-1258(in Chinese)(王蜜宫,陈锻生,林超.基于局部形状图的三维人脸特征点自动定位[J].计算机应用,2010(5):1255-1258)
[27]Dorai C,Jain A K.COSMOS-a representation scheme for 3Dfree-form objects[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1997,19(10):1115-1130
[28]Wuhrer S,Azouz Z B,Shu C.Semi-automatic prediction of landmarks on human models in varying poses[C]//Proceedings of the Canadian Conference on Computer and Robot Vision.Los Alamitos:IEEE Computer Society Press,2010:136-142
[29]Duda R O,Hart P E,and Stork D G.Pattern classification[M].2nd ed.New York:John Wiley&Sons,2000
[30]Robinette K M,Daanen H,Paquet E.The CAESAR project:a3-D surface anthropometry survey[C]//Proceedings of the 2nd International Conference on 3D Digital Imaging and Modeling.Los Alamitos:IEEE Computer Society Press,1999:380-386
[31]Pearl J.Probabilistic reasoning in intelligent systems[J].Computer Science Artificial Intelligence,1988,70(2):1022-1027
[32]King D E.Dlib-ml:a machine learning toolkit[J].Journal of Machine Learning Research,2009,10:1755-1758
[33]Garland M,Heckbert P S.Surface simplification using quadric error metrics[C]//Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques.New York:ACM Press,1997:209-216
[34]Bentley J L.Multidimensional binary search trees used for associative searching[J].Communications of the ACM,1975,18(9):509-517
[35]Pishchulin L,Wuhrer S,Helten T,et al.Building statistical shape spaces for 3D human modeling[J].Pattern Recognition,2017,67:276-286
[36]Field D A.Laplacian smoothing and delaunay triangulations[J].Communications in Applied Numerical Methods,1988,4(6):709-712
[2]Kobbelt L,Campagna S,Vorsatz J,et al.Interactive multiresolution modeling on arbitrary meshes[C]//Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques.New York:ACM Press,1998:105-114
[3]Botsch M,Kobbelt L.An intuitive framework for real-time freeform modeling[J].ACM Transactions on Graphics,2004,23(3):630-634
[4]Allen B,Curless B,Popovic Z.The space of human body shapes:reconstruction and parameterization from range scans[J].ACM Transaction on Graphics,2003,22(3):587-594
[5]Sumner R W,Schmid J,Pauly M.Embedded deformation for shape manipulation[J].ACM Transactions on Graphics,2007,26(3):Article No.80
[6]Goodall C.Procrustes methods in the statistical analysis of shape[J].Journal of the Royal Statistical Society,1991,53(2):285-339
[7]Magnenat-Thalmann N,Laperrire R,Thalmann D.Joint-dependent local deformations for hand animation and object grasping[C]//Proceedings of on Graphics Interface.Toronto:Canadian Information Processing Society Press,1988:26-33
[8]Anguelov D,Srinivasan P,Koller D,et al.SCAPE:shape completion and animation of people[J].ACM Transactions on Graphics,2005,24(3):408-416
[9]Loper M,Mahmood N,Romero J,et al.SMPL:a skinned multi-person linear model[J].ACM Transactions on Graphics,2015,34(6):Article No.248
[10]Li Guiqing,Ma Weiyin,Bao Hujun.Fitting system using loop subdivision surfaces with sharp features[J].Journal of Computer-Aided Design&Computer Graphics,2005,17(6):1179-1185(in Chinese)(李桂清,马维银,鲍虎军.带尖锐特征的Loop细分曲面拟合系统[J].计算机辅助设计与图形学学报,2005,17(6):1179-1185)
[11]Wang Y P,Li G Q,Zeng Z C,et al.Articulated-motion-aware sparse localized decomposition[J].Computer Graphics Forum,2017,36(8):247-259
[12]Hirshberg D A,Loper M,Rachlin E,et al.Coregistration:simultaneous alignment and modeling of articulated 3Dshape[C]//Proceedings of the 12th European Conference on Computer Vision.Heidelherg:Springer,2012:242-255
[13]Zhou Jin,Pan Jianjiang,Tong Jing,et al.Fast 3D human body reconstruction from Kinects[J].Journal of Computer-Aided Design&Computer Graphics,2013,25(6):873-879(in Chinese)(周瑾,潘建江,童晶,等.使用Kinect快速重建三维人体[J].计算机辅助设计与图形学学报,2013,25(6):873-879)
[14]Dong Hongwei.Mesh deformation based 3D facial modeling from images[J].Journal of Computer-Aided Design&Computer Graphics,2012,24(7):932-940(in Chinese)(董洪伟.基于网格变形的从图像重建三维人脸[J].计算机辅助设计与图形学学报,2012,24(7):932-940)
[15]Gu L,Kanade T.3D alignment of face in a single image[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2006:1305-1312
[16]Gou C,Wu Y,Wang F Y,et al.Shape augmented regression for 3D face alignment[C]//Proceedings of European Conference on Computer Vision.Heidelherg:Springer,2016:604-615
[17]Azouz Z B,Shu C,Mantel A.Automatic locating of anthropometric landmarks on 3D human models[C]//Proceedings of the 3rd International Symposium on 3D Data Processing,Visualization,and Transmission.Los Alamitos:IEEE Computer Society Press,2006:750-757
[18]Johnson A E.Spin-Images:a representation for 3-D surface matching[D].Pittsburgh:Carnegie Mellon University,1997
[19]Wuhrer S,Azouz Z B,Shu C.Semi-automatic prediction of landmarks on human models in varying poses[C]//Proceedings of the Canadian Conference on Computer and Robot Vision.Los Alamitos:IEEE Computer Society Press,2010:136-142
[20]Li C Y.Spectral geometric methods for deformable 3D shape retrieval[D].Montreal:Concordia University,2013
[21]Han H,Nam Y.Automatic body landmark identification for various body figures[J].International Journal of Industrial Ergonomics,2011,41(6):592-606
[22]Li Yuelong,Jin Yan,Wang Jianming.Face feature points extraction:a review[J].Chinese Journal of Computers,2016,39(7):1356-1374(in Chinese)(李月龙,靳彦,汪剑鸣.人脸特征点提取方法综述[J].计算机学报,2016,39(7):1356-1374)
[23]Cootes T F,Taylor C J.Statistical models of appearance for computer vision[R].Manchester:University of Manchester,Technical Report,2004
[24]Tong Y,Wang Y,Zhu Z W,et al.Robust facial feature tracking under varying face pose and facial expression[J].Pattern Recognition,2007,40(11):3195-3208
[25]Kazemi V,Sullivan J.One millisecond face alignment with an ensemble of regression trees[C]//Proceedings of the IEEEConference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2014:1867-1874
[26]Wang Migong,Chen Duansheng,Lin Chao.Automatic location of 3D facial feature points based on local shape map[J].Journal of Computer Applications,2010,30(5):1255-1258(in Chinese)(王蜜宫,陈锻生,林超.基于局部形状图的三维人脸特征点自动定位[J].计算机应用,2010(5):1255-1258)
[27]Dorai C,Jain A K.COSMOS-a representation scheme for 3Dfree-form objects[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1997,19(10):1115-1130
[28]Wuhrer S,Azouz Z B,Shu C.Semi-automatic prediction of landmarks on human models in varying poses[C]//Proceedings of the Canadian Conference on Computer and Robot Vision.Los Alamitos:IEEE Computer Society Press,2010:136-142
[29]Duda R O,Hart P E,and Stork D G.Pattern classification[M].2nd ed.New York:John Wiley&Sons,2000
[30]Robinette K M,Daanen H,Paquet E.The CAESAR project:a3-D surface anthropometry survey[C]//Proceedings of the 2nd International Conference on 3D Digital Imaging and Modeling.Los Alamitos:IEEE Computer Society Press,1999:380-386
[31]Pearl J.Probabilistic reasoning in intelligent systems[J].Computer Science Artificial Intelligence,1988,70(2):1022-1027
[32]King D E.Dlib-ml:a machine learning toolkit[J].Journal of Machine Learning Research,2009,10:1755-1758
[33]Garland M,Heckbert P S.Surface simplification using quadric error metrics[C]//Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques.New York:ACM Press,1997:209-216
[34]Bentley J L.Multidimensional binary search trees used for associative searching[J].Communications of the ACM,1975,18(9):509-517
[35]Pishchulin L,Wuhrer S,Helten T,et al.Building statistical shape spaces for 3D human modeling[J].Pattern Recognition,2017,67:276-286
[36]Field D A.Laplacian smoothing and delaunay triangulations[J].Communications in Applied Numerical Methods,1988,4(6):709-712