基于3DMM模型的三维人脸建模方法的实现
|
摘要
随着在人脸识别,3D游戏,3D动画电影,医疗等领域中人脸三维建模应用需求的不断增长,人脸建模技术已经成为计算机视觉领域的研究热点,而3DMM模型更是在建模技术中备受关注的方法之一
本文主要介绍了三维人脸建模的主要方法和应用领域.人脸建模的三维形变模型(3DMM)最早是由Blanz和Vetter等人提出的,这种方法的最大的优点就是自动化程度高且真实感强.因此本文除了简要介绍其它方法外,主要介绍三维形变模型,并实现了基于三维形变模型的人脸重建算法.我们发现,去掉关于纹理的约束,对精度影响不是很大.具体的做法是,我们通过对BJUT 3D Face Database中的人脸三维数据进行预处理,得到了一组人脸基底(basis-faces),然后构造出了关于人脸基底与目标人脸之间关系的代价方程,再通过对方程优化求解,最后得到目标人脸模型.
本文采用了VC++6.0与Matlab混合编程,实现了三维人脸重建算法,只要给定一幅或几幅二维人脸图片,就可以生成对应的目标人脸三维模型.在不影响重建精度的前提下,减少代价方程中的部分约束条件,从而减小了算法的时间复杂度.
Due to the increasing requirement in commercial and legal fields,3D face reconstruction has become one of the most popular research area in computer vision.
Face modeling is widely used in face recognition,3D animated films and games, physics and judicature. However, with the development of 3D face recognition, advanced face reconstruction technology is required, two specific development is to increase efficiency and automation level. It is also a fast-developing area. The currently popular technologies include laser scanning, structure light,3DMM.
Technology such as laser scanning has high accuracy for detail description but processing equipments are too expensive for dissemination, no mention the difficult operation and large volume. As a result it can not be used in research and daily life. A new method called 3DMM was proposed by Blanz and Vetter recently. Its advantages include high automation and reality level.
This method is based on the fact human face can be modeled by linear space, so constructing a group of basis-faces, solving it by optimization thus rebuilding the model of other people's face is possible.
In this paper, the BJUT 3D face database of Beijing University is utilized to construct the three-dimensional basis-faces. The 3D face database data were accumulated by cyberware laser scanner. Because of size differences among hu-man faces, even the feature points(such as eyes, mouth,profile) are difficult to coordinate not to mention the three-dimensional vertex, so pre-processing is necessary in order to achieve well corresponding dense faces models ini- tially. Since the three-dimensional structure of people faces are very complex, Firstly, the three-dimensional data were projected onto two-dimensional plane through cylindrical coordinates parameter to simplify calculation. According to the following formula, the 3D coordinates(X, Y, Z)can be transformed into cylindrical(h, r,φ):
then 2D faces were divided based on facial features points(such as eyes, mouth, ears, etc). In this paper it was divided into 36 areas. After that faces had reached regional correspondence but the points within each area are not. Therefore, they can be interpolated: vj,k→(new)=vj,Pre→(temp)* (1-Δr)+vfol→(ori)*Δr
then the 3D data can be obtained from the formulae above and the faces are densely corresponding. Followed the mentioned instruction all the faces in the BJUT 3D Face Database were processed then basis-faces were constructed.
Next step is defining the cost equation:
Shape Prior:
A minimum problem was established through the shape information:
the database stored information of 500 faces,so n=500.
Landmark:
Landmark(li,pi)contains 3D coordinate li and 2D coordinate pi, and the goal is to diminish the distance between the projected point from 3D faces and 2D points.
Color Difference Cost:
color difference cost means the color difference between two correspond-ing points of two-dimensional images. Several points are selected from the three-dimensional model and projected to two-dimensional image according to demarcation of camera. Based on these color information, cost function can be obtained.
Thus the cost function is:
Which can be solved by Levenberg-Marquardt method.
本文主要介绍了三维人脸建模的主要方法和应用领域.人脸建模的三维形变模型(3DMM)最早是由Blanz和Vetter等人提出的,这种方法的最大的优点就是自动化程度高且真实感强.因此本文除了简要介绍其它方法外,主要介绍三维形变模型,并实现了基于三维形变模型的人脸重建算法.我们发现,去掉关于纹理的约束,对精度影响不是很大.具体的做法是,我们通过对BJUT 3D Face Database中的人脸三维数据进行预处理,得到了一组人脸基底(basis-faces),然后构造出了关于人脸基底与目标人脸之间关系的代价方程,再通过对方程优化求解,最后得到目标人脸模型.
本文采用了VC++6.0与Matlab混合编程,实现了三维人脸重建算法,只要给定一幅或几幅二维人脸图片,就可以生成对应的目标人脸三维模型.在不影响重建精度的前提下,减少代价方程中的部分约束条件,从而减小了算法的时间复杂度.
Due to the increasing requirement in commercial and legal fields,3D face reconstruction has become one of the most popular research area in computer vision.
Face modeling is widely used in face recognition,3D animated films and games, physics and judicature. However, with the development of 3D face recognition, advanced face reconstruction technology is required, two specific development is to increase efficiency and automation level. It is also a fast-developing area. The currently popular technologies include laser scanning, structure light,3DMM.
Technology such as laser scanning has high accuracy for detail description but processing equipments are too expensive for dissemination, no mention the difficult operation and large volume. As a result it can not be used in research and daily life. A new method called 3DMM was proposed by Blanz and Vetter recently. Its advantages include high automation and reality level.
This method is based on the fact human face can be modeled by linear space, so constructing a group of basis-faces, solving it by optimization thus rebuilding the model of other people's face is possible.
In this paper, the BJUT 3D face database of Beijing University is utilized to construct the three-dimensional basis-faces. The 3D face database data were accumulated by cyberware laser scanner. Because of size differences among hu-man faces, even the feature points(such as eyes, mouth,profile) are difficult to coordinate not to mention the three-dimensional vertex, so pre-processing is necessary in order to achieve well corresponding dense faces models ini- tially. Since the three-dimensional structure of people faces are very complex, Firstly, the three-dimensional data were projected onto two-dimensional plane through cylindrical coordinates parameter to simplify calculation. According to the following formula, the 3D coordinates(X, Y, Z)can be transformed into cylindrical(h, r,φ):
then 2D faces were divided based on facial features points(such as eyes, mouth, ears, etc). In this paper it was divided into 36 areas. After that faces had reached regional correspondence but the points within each area are not. Therefore, they can be interpolated: vj,k→(new)=vj,Pre→(temp)* (1-Δr)+vfol→(ori)*Δr
then the 3D data can be obtained from the formulae above and the faces are densely corresponding. Followed the mentioned instruction all the faces in the BJUT 3D Face Database were processed then basis-faces were constructed.
Next step is defining the cost equation:
Shape Prior:
A minimum problem was established through the shape information:
the database stored information of 500 faces,so n=500.
Landmark:
Landmark(li,pi)contains 3D coordinate li and 2D coordinate pi, and the goal is to diminish the distance between the projected point from 3D faces and 2D points.
Color Difference Cost:
color difference cost means the color difference between two correspond-ing points of two-dimensional images. Several points are selected from the three-dimensional model and projected to two-dimensional image according to demarcation of camera. Based on these color information, cost function can be obtained.
Thus the cost function is:
Which can be solved by Levenberg-Marquardt method.
引文
[1]The BJUT-3D Large-Scale Chinese Face Database[DB/OL]. http://www.bjut.edu.cn/sci/multimedia/mul-lab/3dface/face_database.htm
[2]Brian Amberg, Andrew Blake, Andrew Fitzgibbon, Sami Romdhani, Thomas Vetter. Reconstructing High Quality Face-Surfaces using Model Based Stereo[C]. ICCV 2007,2007.
[3]Volker Blanz, Thomas Vetter. A Morphable model for the synthesis of 3D faces[C]. In Proceedings, SIGGRAPH'99,1999:187-194
[4]胡永利.真实感三维人脸建模及应用研究[D].北京：北京工业大学计算机学院,2004.
[5]Hui Guo, Jiayan Jiang, Liming Zhang. Building a 3D Morphable Face Model by Using Thin Plate Splines for Face Reconstruction[C]. Sinobiometrics 2004, LNCS 3338,2004:258-267.
[6]Li Feng, Jianhuang Lai, Lei Zhang.3D Surface Reconstruction Based on One Non-symmetric Face Image[C]. Sinobiometrics 2004, LNCS 3338,2004:268-274.
[7]Shu-Fan Wang, Shang-Hong Lai. Efficient 3D Face Reconstruction from a Single 2D Image by Combining Statistical and Geometrical Information[C]. ACCV (2006),2006:427-436.
[8]Changhu Wang, Shuicheng Yan, Hua Li, Hongjiang Zhang, Mingjing Li. Au-tomatic, Effective, and Efficient 3D Face Reconstruction from Arbitrary View Image[C]. PCM 2004, LNCS 3332,2004:553-560.
[9]Daria Kalinkina, Andre Gagalowicz, Richard Roussel2.3D Reconstruction of a Human Face from Images Using Morphological Adaptation[C]. MIRAGE 2007, LNCS 4418,2007:212-224,
[10]Gary A. Atkinson, Abdul R. Farooq, Melvyn L. Smith, Lyndon N. Smith. Facial Reconstruction and Alignment Using Photometric Stereo and Surface Fitting [C]. in Proc. Iberian Conference on Pattern Recognition and Image Anal-ysis, Povoa de Varzim, Portugal 2009,2009:88-95.
[11]Ching Y. SuenA, Arash Zaryabi Langaroudi, Chunghua Feng, Yuxing Mao. A survey of techniques for Face Reconstruction[C]. In Proceedings of International Symposium on Integrated Circuits,Singapore,2007.
[12]H. Fang, N. P. Costen.3D Face Reconstruction Under Imperfect Tracking Circumstances Using Shape Model Constraints[C]. In Proc AVC,2007:519-528.
[13]Xun Gong, Guoyin Wang. An Automatic Approach for Pixel-wise Corre-spondence between 3D Faces [C].2006 International Conference on Hybrid Information Technology,2006.
[14]Thomas Vetter, Volker Blanz. Estimating coloured 3D face models from sin-gle images:An example based approach[C]. Computer Vision-ECCV'98, 1998:499-513.
[15]徐琳,袁保宗,高文.真实感人脸建模研究的进展与展望[J].软件学报,2002.
[16]舒新峰.三维人脸建模技术研究[D].西北大学研究生院,2002.
[17]Xun Gong, Guoyin Wang. A Dynamic Component Deforming Model for Face Shape Reconstruction[C]. ISVC 2007, Part I, LNCS 4841,2007:488-497.
[18]龚勋.基于单张二维图片的三维人脸建模[D].西南交通大学研究生院,2008.
[19]Jesus P. Mena-Chalco, Ives Macedo, Luiz Velho, Roberto M. Cesar Jr.3D face computational photography using PCA spaces[J]. Vis Comput (2009), 2009:899-909.
[20]Abdelrehim Ahmed, Aly Farag. A New Statistical Model Combining Shape and Spherical Harmonics Illumination for Face Reconstruction[C]. ISVC 2007, Part Ⅰ, LNCS 4841,2000:531-541.
[21]Ching Y. Suen, Arash Zaryabi Langaroudi, Chunghua Feng, Yuxing Mao. A survey of techniques for Face Reconstruction [C]. Proceedings of International Symposium on Integrated Circuits,Singapore,2007.
[22]·徐成华,王蕴红,谭铁牛.三维人脸建模与应用.中国图象图形学报[J],2004.
[23]W. ZHAO, R. CHELLAPPA, P. J. PHILLIPS, A. ROSENFELD. Face Recog-nition:A Literature Survey[C]. ACM Computing Surveys, Vol.35, No.4, December 2003,2003:399-458.
[24]Xun Gong, Guoyin Wang. A Novel Deformation Framework for Face Modeling from a Few Control Points[C]. RSKT 2008, LNAI 5009,2008:434-441.
[25]M. Z. BROWN, D. BURSCHKA. Advances in computational stereo[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2003,25(8):993-1008.
[26]袁亚湘.孙文瑜.最优化理论与方法[M].科学出版社,1997.
[2]Brian Amberg, Andrew Blake, Andrew Fitzgibbon, Sami Romdhani, Thomas Vetter. Reconstructing High Quality Face-Surfaces using Model Based Stereo[C]. ICCV 2007,2007.
[3]Volker Blanz, Thomas Vetter. A Morphable model for the synthesis of 3D faces[C]. In Proceedings, SIGGRAPH'99,1999:187-194
[4]胡永利.真实感三维人脸建模及应用研究[D].北京：北京工业大学计算机学院,2004.
[5]Hui Guo, Jiayan Jiang, Liming Zhang. Building a 3D Morphable Face Model by Using Thin Plate Splines for Face Reconstruction[C]. Sinobiometrics 2004, LNCS 3338,2004:258-267.
[6]Li Feng, Jianhuang Lai, Lei Zhang.3D Surface Reconstruction Based on One Non-symmetric Face Image[C]. Sinobiometrics 2004, LNCS 3338,2004:268-274.
[7]Shu-Fan Wang, Shang-Hong Lai. Efficient 3D Face Reconstruction from a Single 2D Image by Combining Statistical and Geometrical Information[C]. ACCV (2006),2006:427-436.
[8]Changhu Wang, Shuicheng Yan, Hua Li, Hongjiang Zhang, Mingjing Li. Au-tomatic, Effective, and Efficient 3D Face Reconstruction from Arbitrary View Image[C]. PCM 2004, LNCS 3332,2004:553-560.
[9]Daria Kalinkina, Andre Gagalowicz, Richard Roussel2.3D Reconstruction of a Human Face from Images Using Morphological Adaptation[C]. MIRAGE 2007, LNCS 4418,2007:212-224,
[10]Gary A. Atkinson, Abdul R. Farooq, Melvyn L. Smith, Lyndon N. Smith. Facial Reconstruction and Alignment Using Photometric Stereo and Surface Fitting [C]. in Proc. Iberian Conference on Pattern Recognition and Image Anal-ysis, Povoa de Varzim, Portugal 2009,2009:88-95.
[11]Ching Y. SuenA, Arash Zaryabi Langaroudi, Chunghua Feng, Yuxing Mao. A survey of techniques for Face Reconstruction[C]. In Proceedings of International Symposium on Integrated Circuits,Singapore,2007.
[12]H. Fang, N. P. Costen.3D Face Reconstruction Under Imperfect Tracking Circumstances Using Shape Model Constraints[C]. In Proc AVC,2007:519-528.
[13]Xun Gong, Guoyin Wang. An Automatic Approach for Pixel-wise Corre-spondence between 3D Faces [C].2006 International Conference on Hybrid Information Technology,2006.
[14]Thomas Vetter, Volker Blanz. Estimating coloured 3D face models from sin-gle images:An example based approach[C]. Computer Vision-ECCV'98, 1998:499-513.
[15]徐琳,袁保宗,高文.真实感人脸建模研究的进展与展望[J].软件学报,2002.
[16]舒新峰.三维人脸建模技术研究[D].西北大学研究生院,2002.
[17]Xun Gong, Guoyin Wang. A Dynamic Component Deforming Model for Face Shape Reconstruction[C]. ISVC 2007, Part I, LNCS 4841,2007:488-497.
[18]龚勋.基于单张二维图片的三维人脸建模[D].西南交通大学研究生院,2008.
[19]Jesus P. Mena-Chalco, Ives Macedo, Luiz Velho, Roberto M. Cesar Jr.3D face computational photography using PCA spaces[J]. Vis Comput (2009), 2009:899-909.
[20]Abdelrehim Ahmed, Aly Farag. A New Statistical Model Combining Shape and Spherical Harmonics Illumination for Face Reconstruction[C]. ISVC 2007, Part Ⅰ, LNCS 4841,2000:531-541.
[21]Ching Y. Suen, Arash Zaryabi Langaroudi, Chunghua Feng, Yuxing Mao. A survey of techniques for Face Reconstruction [C]. Proceedings of International Symposium on Integrated Circuits,Singapore,2007.
[22]·徐成华,王蕴红,谭铁牛.三维人脸建模与应用.中国图象图形学报[J],2004.
[23]W. ZHAO, R. CHELLAPPA, P. J. PHILLIPS, A. ROSENFELD. Face Recog-nition:A Literature Survey[C]. ACM Computing Surveys, Vol.35, No.4, December 2003,2003:399-458.
[24]Xun Gong, Guoyin Wang. A Novel Deformation Framework for Face Modeling from a Few Control Points[C]. RSKT 2008, LNAI 5009,2008:434-441.
[25]M. Z. BROWN, D. BURSCHKA. Advances in computational stereo[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2003,25(8):993-1008.
[26]袁亚湘.孙文瑜.最优化理论与方法[M].科学出版社,1997.