真实感三维人脸建模及表情动画技术的研究
|
摘要
人脸是人类相互表达情感的重要部分。在计算机虚拟世界中，虚拟人脸是虚拟人表情达意的载体。人脸的造型和表情动画作为虚拟现实技术和计算机图形学中的一个分支已有30年历史了，随着影视技术、游戏娱乐、虚拟环境、远程通信、辅助医疗、刑事鉴别等方面应用的日益广泛，这一研究领域越来越受到人们的关注。假如以往的趋势为未来的发展指明了方向的话，接下来的几年对人脸造型和表情的研究将会更令人兴奋!
顺应这样的时代潮流，本论文对真实感人脸的三维建模技术及表情动画技术进行了研究。论文的工作和成果主要有：
1．在真实感三维人脸建模方面
提出了一种简单有效的由正、侧面照片重建特定人脸三维模型的新方法。采用多分辨率技术构造纹理无缝拼接图，然后把三维网格模型投影到圆柱面上并展开，根据二维图象变形技术中特征点匹配算法的数学思想，将匹配公式进行改进后应用于模型网格点的适配，在作了一定假设后推导出逆向圆柱映射的计算公式，再将适配后的柱面网格映射回去，最后进行纹理映射生成真实感的特定人脸的三维模型。
2．在人脸表情动画方面：
基于Waters的几何肌肉模型定制了一个具有17个肌肉向量的人脸模型的动画结构，用非线性插值来模拟肌肉的简单运动；同时，采用旋转变换来模拟其它的表情控制参数，例如下巴的张开角、眼帘的张开角等。从而简单而有效地实现了人脸表情。
由于人脸造型和表情动画技术研究的难度和学位论文工作的时限等原因，上述工作和成果只是局部和基础性的，但其有效性和实用价值也是可见的。
Human face is the window of communication in real world, and virtual face is of the same importance in virtual world. Therefore, the construction and animation of realistic human facial models is always attractive to researchers in computer graphics, and as an identifiable area of the latter in the past 30 years, facial modeling and animation has been devoted to significant efforts with many dozens of research papers published. Its applications include such diverse fields as character animation for films and advertising, computer games, virtual reality, teleconferencing, user-interface, surgical facial planning, and criminal discrimination. Yet no perfectly realistic facial model and animation has ever been generated by computer, it remains one of the most fundermental, and as well the most difficult problem in computer graphics.
The goal of the thesis work is to automate the challenging task of creating realistic facial models of individuals and generating basic expression animation.
Modeling: This thesis develops a method different from traditional techniques that begins with the cylindrical projection of a generic model and its seamless global texture map using multiresolution technique, and then automatically fit the unwrapped cylindrical projected mesh to the texture map with an improved algorithm that based on 2D morphing to specify corresponding feature points(or lines). Finally, we propose a formula of inverse cylindrical projection to recompose the deformed mesh, and after the texture mapping a photo-realistic individual facial model is created.
Expression animation: Based on Water's muscle model, 17 muscle vectors are presented to simulate facial muscles' motion by nonlinear interpolation. Meanwhile, some other parameters, for example, open of jaw and eyelids, could simply be substituted by rotation transform. Through these control parameters' interaction, highly natural looking animations have been able to generated.
The work and achievement in this thesis is just fundamental and pilot study, while it can be exploited in many applications such as natural and
intelligent human-machine interfaces, virtual collaboration work, virtual learning and teaching, and so on.
顺应这样的时代潮流，本论文对真实感人脸的三维建模技术及表情动画技术进行了研究。论文的工作和成果主要有：
1．在真实感三维人脸建模方面
提出了一种简单有效的由正、侧面照片重建特定人脸三维模型的新方法。采用多分辨率技术构造纹理无缝拼接图，然后把三维网格模型投影到圆柱面上并展开，根据二维图象变形技术中特征点匹配算法的数学思想，将匹配公式进行改进后应用于模型网格点的适配，在作了一定假设后推导出逆向圆柱映射的计算公式，再将适配后的柱面网格映射回去，最后进行纹理映射生成真实感的特定人脸的三维模型。
2．在人脸表情动画方面：
基于Waters的几何肌肉模型定制了一个具有17个肌肉向量的人脸模型的动画结构，用非线性插值来模拟肌肉的简单运动；同时，采用旋转变换来模拟其它的表情控制参数，例如下巴的张开角、眼帘的张开角等。从而简单而有效地实现了人脸表情。
由于人脸造型和表情动画技术研究的难度和学位论文工作的时限等原因，上述工作和成果只是局部和基础性的，但其有效性和实用价值也是可见的。
Human face is the window of communication in real world, and virtual face is of the same importance in virtual world. Therefore, the construction and animation of realistic human facial models is always attractive to researchers in computer graphics, and as an identifiable area of the latter in the past 30 years, facial modeling and animation has been devoted to significant efforts with many dozens of research papers published. Its applications include such diverse fields as character animation for films and advertising, computer games, virtual reality, teleconferencing, user-interface, surgical facial planning, and criminal discrimination. Yet no perfectly realistic facial model and animation has ever been generated by computer, it remains one of the most fundermental, and as well the most difficult problem in computer graphics.
The goal of the thesis work is to automate the challenging task of creating realistic facial models of individuals and generating basic expression animation.
Modeling: This thesis develops a method different from traditional techniques that begins with the cylindrical projection of a generic model and its seamless global texture map using multiresolution technique, and then automatically fit the unwrapped cylindrical projected mesh to the texture map with an improved algorithm that based on 2D morphing to specify corresponding feature points(or lines). Finally, we propose a formula of inverse cylindrical projection to recompose the deformed mesh, and after the texture mapping a photo-realistic individual facial model is created.
Expression animation: Based on Water's muscle model, 17 muscle vectors are presented to simulate facial muscles' motion by nonlinear interpolation. Meanwhile, some other parameters, for example, open of jaw and eyelids, could simply be substituted by rotation transform. Through these control parameters' interaction, highly natural looking animations have been able to generated.
The work and achievement in this thesis is just fundamental and pilot study, while it can be exploited in many applications such as natural and
intelligent human-machine interfaces, virtual collaboration work, virtual learning and teaching, and so on.
引文
[1] 梅丽，鲍虎军，彭群生：特定人脸的快速定制和肌肉驱动的表情动画，计算机辅助设计与图形学学报，13(12)：1077-1082，2001
[2] Kiyoshi Arai, Tsuneya Kurihara, Ken-ichi Anjyo: Bilinear interpolation for facial expression and metamorphosis in real-time animation, The Visual Computer, 12:105-116, 1996
[3] Keith Waters: A Muscle Model for Animating Three-Dimensional Facial Expression, Computer Graphics, 21(4):17-24, 1987
[4] Yuencheng Lee, Demetri Terzopoulos, Keith Waters: Realistic Modeling for Facial Animation, In: Computer Graphics Proceedings, Annual Conference Series, 55-62, ACM SIGGRAPH, August 1995
[5] P Ekman, W. V Friesen: Facial Action Coding System, onsulting Psychologists Press, Palo Alto, CA, 1978
[6] M. Oka, K. Tsutsui, A. ohba, Y. Jurauchi, T. Tago: Real-time manipulation of-texture-mapped surface, In: Computer Graphics Proceedings, Annual Conference Series, 181-188, ACM SIGGRAPH, 1987
[7] P. Kalra, N. Magnenat-Thanmann: Modeling of Vascular Expressions in Facial Animation, Computer Animation, 50-58, 1994
[8] Bier T, Neely S: Feature-Based Image Metamorphosis, In: Computer Graphics Proceedings, Annual Conference Series, 35-42, ACM SIGGRAPH, July 1992
[9] Bergeron P.: Techniques for animating characters, In: Computer Graphics Proceedings, Annual Conference Series, ACM SIGGRAPH, 1985
[10] Zicheng Liu, Zhengyou Zhang, Chuck Jacobs, Michael Cohen: Rapid Modeling of Animated Faces From Video, Microsoft Research Technical Report, 99-21, April 1999
[11] Frederic Pighin, Jamie Hecker, Dani Lischinski, Richard Szeliski, David H. Salesin: Synthesizing Realistic Facial Expressions from Photographs, In: Computer Graphics Proceedings, Annual Conference Series, 75-84, ACM SIGGRAPH, July 1998
[12] Gregory M. Nielson: Scattered Data Modeling, IEEE Computer Graphics & Applications, 13(1):60-70, 1993
[13] Takaaki Akimoto, Yasuhito Suenaga, Richard S. Wallace: Automatic Creation of 3D Facial Models, IEEE Computer Graphics & Applications, 13(5):16-22, 1993
[14] Frederic Pighin, Joel Auslander, Dani Lischinski, David H. Salesin, Richard Szeliski: Realistic Facial Animation Using Image-Based 3D Morphing, Technical report, UW-CSE-97-01-03, 1997
[15] David T. Chen, Andrei State, David Banks: Interactive Shape Metamorphosis, In: 1995 Symposium on Interactive 3D Graphics, 43-44, ACM SIGGRAPH, 1995
[16] D. Terzopoulos, K. Waters: Analysis and synthesis of facial image sequences using physical and anatomical models, In: IEEE Transactions on Pattern Analysis and Machine Intelligence, 569-579, June 1993
[17] Lijun Yin, Anup Basu MPEG4 Face Modeling Using Fiducial Points Department of Computing Science, University of Alberta, Canada
[18] Horace H.S.Ip, Linjun Yin: Constructing a 3D individualized head model from two orthogonal views, The Visual Computer, 12:254-266, 1996
[19] 舒新峰：三维人脸建模技术研究，硕士学位论文，西北大学，2002
[20] 任惠民，张怀瑶：人体系统解剖学，陕西科学技术出版社，1991
[21] 张汗灵：基于图像的绘制关键技术研究，博士学位论文，西北工业大学，2003
[22] 崔屹：数字图像处理技术与应用，电子工业出版社，北京，1997
[23] 张翔宇，林志勇，华蓓，陈意云：OpenGL在人脸建模和动画中的应用，计算机应用，20(8)：33-37，2000．
[24] 张翔宇，林志勇，华蓓，陈意云：从正面侧面照片合成三维人脸，计算机应用，20(7)：41-45，2000
[25] 晏洁，高文：基于一般人脸模型修改的特定人脸合成技术，计算机辅助设计与图形学学报，11(5)：389-394，1999
[26] 虞露：MPEG-4中脸部动画参数和序列重绘的肌肉模型，中国图象图形学报，6(1)：36-41，2001
[27] 王奎武，董兰芳，王润，陈意云：基于MPEG-4的人脸变形算法的研究，计算机辅助设计与图形学学报，14(1)：21-25，2002
[28] 王强：面向MPEG4的人脸建模方法和工具，硕士学位论文，清华大学，2000．11
[29] 张翔宇，华蓓，陈意云：人脸建模和动画的基本技术，计算机辅助设计与图形学学报，13(4)：342-347，2001
[30] 晏洁，高文，尹宝才：具有真实感的三维虚拟特定人脸生成方法，计算机学报，22(2)：147-153，1999
[31] 王瑜：OpenGL在三维造型中的应用，硕士学位论文，大连理工大学，2000
[32] 梅丽：基于图像的脸部重建和视频驱动的表情动画，博士学位论文，浙江大学，2001
[33] 乔林，费广正，林杜等：OpenGL程序设计，清华大学出版社，2000
[34] H. Sera, S. Morishma, D. Terzopoulos: Physics-based Muscle Model for Moth Shape Control, IEEE International Workshop on Robot and Human Communication, 207-212, 1996
[35] K. Waters, T. M. Levergood, Decface: An Automatic Lip-Synchronization Algorithm for Synthetic Faces, Cambridge Research Laboratory Technical Report Series, December 1993
[36] Kass M, Witkin A, Terzopoulos D: Snakes: active contour models, International Journal of Computer Vision, 1(4):321-331, 1988
[37] Williams, et al: A fast algorithm for active contour and curvature estimation, CVGIP: Image Understanding, 55(1):14-26, 1992
[38] Marc Proesmans, Luc Van Gool: Reading between the lines-a method for extracting dynamic 3D with texture, In: Proceedings of VRST, 95-102, 1997
[39] Brian Guenter, Cindy Grimm, Daniel Wood, Henrique Malvar, Fredrick Pighin: Making Faces, In: Computer Graphics Proceedings, Annual Conference Series, 55-66, ACM SIGGRAPH, July 1998
[40] N. Magnenat-Thalmann, H.T.Minh, M. de Angelis, D. Thalmann: Design,transformation and animation of human faces, The Visual Computer, 5:32-39, 1989
[41] P. Fau, C. Miccio: Animated Heads from Ordinary Images: A Least-Squares Approach, Computer Vision and Image Understanding, 75(3):247-259, 1999
[42] Parke F I: Computer Generated Animation of Faces, Master's thesis, University of Utah, Salt Lake City, UTEC-CSc-72-120, December 1972
[43] Parke F I: A parameterization model for human faces, PhD dissertation, University of Utah, Salt Lake City, UT, 1974
[44] Won-Sook Lee, Nadia Magnenat-Thalmann: Fast Head Modeling for Animation, Journal Image and Vision Computing, 18(4): 355-364, Elsevier,
1, March, 2000
[45] Peter J Burt, Edward H Andelson: A multiresolution spline with application to image mosaics, ACM Transactions on Graphics, 2(4):217-236,1983
[46] James R. Kent, Wayne E. Carlson, Richard E. Parent: Shape Transformation for Polyhedral Objects, Computer Graphics, 26(2):47-54, 1992
[47] Douglas DeCarlo, Dimitris Metaxas, Matthrew Stone: An Anthropometric Face Model using Variational Techniques, In: Computer Graphics Proceedings, Annual Conference Series, 67-74, ACM SIGGRAPH, July 1998
[48] Steven M. Seitz, Charles R. Dyer: View Morphing, In: Computer Graphics Proceedings, Annual Conference Series, 21-30, ACM SIGGRAPH, August 1996
[49] Paul E. Debevec, Camillo J. Taylor, Jitendra Malik: Modeling and Rendering Architecture from Photographs: A hybrid geometry-and image-based approach, In: Computer Graphics Proceedings, Annual Conference Series, 11-20, ACM SIGGRAPH, August 1996
[50] Pierre Badin, Gerard Bailly, Lionel Reveret: Three-dimensional linear articulatory modeling of tongue, lips and face, based on MRI and video images, Journal of Phonetics, (2002) 30, 533-553, 2002
[51] Isaac Rudomin, Adriana Boj(?)rquez, H(?)ctor Cuevas: Statistical Generation of 3d Facial Animable Models, Proceedings of the Shape Modeling International (SMI'02), 2002 IEEE
[52] Diana Phillips Mahoney: Making Faces, Computer Graphics World, Nov. 2000
[53] Jiang Dalong, Li Zhiguo, Wang Zhaoqi, Gao Wen: Animating 3D Facial Models with MPEG-4 FaceDefTables, Proceedings of the 35~th Annual Simulation Symposium (SS'02), 2002 IEEE
[54] Tony DeRose, Michael Kass, Tien Truong: Subdivision Surfaces in Character Animation, In: Computer Graphics Proceedings, Annual Conference Series, 85-94, ACM SIGGRAPH, July 1998
[55] S. M. Platt, N. Badler: Animating facial expression, Computer Graphics, 15(3): 245-252, 1981
[56] P. Kalra, A. Mangili, N. M. Thalmann, D. Thalmann: Simulation of Facial
Muscle Action Based on Rational Free Form Deformations, Eurographics,11(3): 59-69, 1992
[2] Kiyoshi Arai, Tsuneya Kurihara, Ken-ichi Anjyo: Bilinear interpolation for facial expression and metamorphosis in real-time animation, The Visual Computer, 12:105-116, 1996
[3] Keith Waters: A Muscle Model for Animating Three-Dimensional Facial Expression, Computer Graphics, 21(4):17-24, 1987
[4] Yuencheng Lee, Demetri Terzopoulos, Keith Waters: Realistic Modeling for Facial Animation, In: Computer Graphics Proceedings, Annual Conference Series, 55-62, ACM SIGGRAPH, August 1995
[5] P Ekman, W. V Friesen: Facial Action Coding System, onsulting Psychologists Press, Palo Alto, CA, 1978
[6] M. Oka, K. Tsutsui, A. ohba, Y. Jurauchi, T. Tago: Real-time manipulation of-texture-mapped surface, In: Computer Graphics Proceedings, Annual Conference Series, 181-188, ACM SIGGRAPH, 1987
[7] P. Kalra, N. Magnenat-Thanmann: Modeling of Vascular Expressions in Facial Animation, Computer Animation, 50-58, 1994
[8] Bier T, Neely S: Feature-Based Image Metamorphosis, In: Computer Graphics Proceedings, Annual Conference Series, 35-42, ACM SIGGRAPH, July 1992
[9] Bergeron P.: Techniques for animating characters, In: Computer Graphics Proceedings, Annual Conference Series, ACM SIGGRAPH, 1985
[10] Zicheng Liu, Zhengyou Zhang, Chuck Jacobs, Michael Cohen: Rapid Modeling of Animated Faces From Video, Microsoft Research Technical Report, 99-21, April 1999
[11] Frederic Pighin, Jamie Hecker, Dani Lischinski, Richard Szeliski, David H. Salesin: Synthesizing Realistic Facial Expressions from Photographs, In: Computer Graphics Proceedings, Annual Conference Series, 75-84, ACM SIGGRAPH, July 1998
[12] Gregory M. Nielson: Scattered Data Modeling, IEEE Computer Graphics & Applications, 13(1):60-70, 1993
[13] Takaaki Akimoto, Yasuhito Suenaga, Richard S. Wallace: Automatic Creation of 3D Facial Models, IEEE Computer Graphics & Applications, 13(5):16-22, 1993
[14] Frederic Pighin, Joel Auslander, Dani Lischinski, David H. Salesin, Richard Szeliski: Realistic Facial Animation Using Image-Based 3D Morphing, Technical report, UW-CSE-97-01-03, 1997
[15] David T. Chen, Andrei State, David Banks: Interactive Shape Metamorphosis, In: 1995 Symposium on Interactive 3D Graphics, 43-44, ACM SIGGRAPH, 1995
[16] D. Terzopoulos, K. Waters: Analysis and synthesis of facial image sequences using physical and anatomical models, In: IEEE Transactions on Pattern Analysis and Machine Intelligence, 569-579, June 1993
[17] Lijun Yin, Anup Basu MPEG4 Face Modeling Using Fiducial Points Department of Computing Science, University of Alberta, Canada
[18] Horace H.S.Ip, Linjun Yin: Constructing a 3D individualized head model from two orthogonal views, The Visual Computer, 12:254-266, 1996
[19] 舒新峰：三维人脸建模技术研究，硕士学位论文，西北大学，2002
[20] 任惠民，张怀瑶：人体系统解剖学，陕西科学技术出版社，1991
[21] 张汗灵：基于图像的绘制关键技术研究，博士学位论文，西北工业大学，2003
[22] 崔屹：数字图像处理技术与应用，电子工业出版社，北京，1997
[23] 张翔宇，林志勇，华蓓，陈意云：OpenGL在人脸建模和动画中的应用，计算机应用，20(8)：33-37，2000．
[24] 张翔宇，林志勇，华蓓，陈意云：从正面侧面照片合成三维人脸，计算机应用，20(7)：41-45，2000
[25] 晏洁，高文：基于一般人脸模型修改的特定人脸合成技术，计算机辅助设计与图形学学报，11(5)：389-394，1999
[26] 虞露：MPEG-4中脸部动画参数和序列重绘的肌肉模型，中国图象图形学报，6(1)：36-41，2001
[27] 王奎武，董兰芳，王润，陈意云：基于MPEG-4的人脸变形算法的研究，计算机辅助设计与图形学学报，14(1)：21-25，2002
[28] 王强：面向MPEG4的人脸建模方法和工具，硕士学位论文，清华大学，2000．11
[29] 张翔宇，华蓓，陈意云：人脸建模和动画的基本技术，计算机辅助设计与图形学学报，13(4)：342-347，2001
[30] 晏洁，高文，尹宝才：具有真实感的三维虚拟特定人脸生成方法，计算机学报，22(2)：147-153，1999
[31] 王瑜：OpenGL在三维造型中的应用，硕士学位论文，大连理工大学，2000
[32] 梅丽：基于图像的脸部重建和视频驱动的表情动画，博士学位论文，浙江大学，2001
[33] 乔林，费广正，林杜等：OpenGL程序设计，清华大学出版社，2000
[34] H. Sera, S. Morishma, D. Terzopoulos: Physics-based Muscle Model for Moth Shape Control, IEEE International Workshop on Robot and Human Communication, 207-212, 1996
[35] K. Waters, T. M. Levergood, Decface: An Automatic Lip-Synchronization Algorithm for Synthetic Faces, Cambridge Research Laboratory Technical Report Series, December 1993
[36] Kass M, Witkin A, Terzopoulos D: Snakes: active contour models, International Journal of Computer Vision, 1(4):321-331, 1988
[37] Williams, et al: A fast algorithm for active contour and curvature estimation, CVGIP: Image Understanding, 55(1):14-26, 1992
[38] Marc Proesmans, Luc Van Gool: Reading between the lines-a method for extracting dynamic 3D with texture, In: Proceedings of VRST, 95-102, 1997
[39] Brian Guenter, Cindy Grimm, Daniel Wood, Henrique Malvar, Fredrick Pighin: Making Faces, In: Computer Graphics Proceedings, Annual Conference Series, 55-66, ACM SIGGRAPH, July 1998
[40] N. Magnenat-Thalmann, H.T.Minh, M. de Angelis, D. Thalmann: Design,transformation and animation of human faces, The Visual Computer, 5:32-39, 1989
[41] P. Fau, C. Miccio: Animated Heads from Ordinary Images: A Least-Squares Approach, Computer Vision and Image Understanding, 75(3):247-259, 1999
[42] Parke F I: Computer Generated Animation of Faces, Master's thesis, University of Utah, Salt Lake City, UTEC-CSc-72-120, December 1972
[43] Parke F I: A parameterization model for human faces, PhD dissertation, University of Utah, Salt Lake City, UT, 1974
[44] Won-Sook Lee, Nadia Magnenat-Thalmann: Fast Head Modeling for Animation, Journal Image and Vision Computing, 18(4): 355-364, Elsevier,
1, March, 2000
[45] Peter J Burt, Edward H Andelson: A multiresolution spline with application to image mosaics, ACM Transactions on Graphics, 2(4):217-236,1983
[46] James R. Kent, Wayne E. Carlson, Richard E. Parent: Shape Transformation for Polyhedral Objects, Computer Graphics, 26(2):47-54, 1992
[47] Douglas DeCarlo, Dimitris Metaxas, Matthrew Stone: An Anthropometric Face Model using Variational Techniques, In: Computer Graphics Proceedings, Annual Conference Series, 67-74, ACM SIGGRAPH, July 1998
[48] Steven M. Seitz, Charles R. Dyer: View Morphing, In: Computer Graphics Proceedings, Annual Conference Series, 21-30, ACM SIGGRAPH, August 1996
[49] Paul E. Debevec, Camillo J. Taylor, Jitendra Malik: Modeling and Rendering Architecture from Photographs: A hybrid geometry-and image-based approach, In: Computer Graphics Proceedings, Annual Conference Series, 11-20, ACM SIGGRAPH, August 1996
[50] Pierre Badin, Gerard Bailly, Lionel Reveret: Three-dimensional linear articulatory modeling of tongue, lips and face, based on MRI and video images, Journal of Phonetics, (2002) 30, 533-553, 2002
[51] Isaac Rudomin, Adriana Boj(?)rquez, H(?)ctor Cuevas: Statistical Generation of 3d Facial Animable Models, Proceedings of the Shape Modeling International (SMI'02), 2002 IEEE
[52] Diana Phillips Mahoney: Making Faces, Computer Graphics World, Nov. 2000
[53] Jiang Dalong, Li Zhiguo, Wang Zhaoqi, Gao Wen: Animating 3D Facial Models with MPEG-4 FaceDefTables, Proceedings of the 35~th Annual Simulation Symposium (SS'02), 2002 IEEE
[54] Tony DeRose, Michael Kass, Tien Truong: Subdivision Surfaces in Character Animation, In: Computer Graphics Proceedings, Annual Conference Series, 85-94, ACM SIGGRAPH, July 1998
[55] S. M. Platt, N. Badler: Animating facial expression, Computer Graphics, 15(3): 245-252, 1981
[56] P. Kalra, A. Mangili, N. M. Thalmann, D. Thalmann: Simulation of Facial
Muscle Action Based on Rational Free Form Deformations, Eurographics,11(3): 59-69, 1992