基于样例的表情生成方法研究
|
摘要
基于样例的表情生成是指利用一对样例表情人脸,通过表情计算生成一致的目标表情人脸。通过基于样例人脸的表情生成方法研究,探讨人脸建模、表情映射和纹理贴图等关键技术,对深入研究逼真的表情合成方法具有重要的意义。此外,表情生成方法在智能人机交互、计算机动漫、数字娱乐等领域具有广泛的应用价值。?
论文的主要工作和创新如下:?
首先,本文采用CANDIDE‐3模型作为标准的人脸网格模型。通过基于OpenGL的特征点拾取技术的人机交互方式对CANDIDE‐3三角网格模型上顶点的进行调整后,实现了CANDIDE‐3标准模型与特定人脸的匹配,最后得到了一个符合特定人脸特征的三角网格模型。?
其次,对样例人脸的三角网格模型进行倾斜度校正等预处理后,本文实现的基于样例的表情映射算法把样例人脸的表情运动数据,映射到目标人脸网格模型上,计算出目标人脸相应表情的三角网格模型。由于表情映射后网格模型上存在一些奇异点,本文对这个网格模型添加了两个约束,一是对模型上网格点的运动范围进行约束,二是依据人脸的对称性对网格模型上的网格点添加对称性约束。?
第三,基于OpenGL纹理映射技术的图像变形算法,对表情映射生成的三角网格模型进行纹理贴图。本算法依据目标人脸的中性表情图像、中性表情对应的人脸三角网格模型和表情映射后的三角网格模型数据,得到了目标人脸的逼真的不走样的表情图像。?
最后,本文对所提出的表情生成算法的相关步骤逐步进行了一系列实验,最后对整体算法给出了综合实验和交叉验证实验,验证了算法的有效性。?
Example-Based Expression Analogy can be defined as: given an neutral expression image and its another expression image, expression blending retargets the sample's expression motion data to another neutral expression image and gets its another expression image.
My work, example-based expression analogy, mainly including human face modeling, expression transformation, texture mapping, is of significance to purse further study on expression morph. Moreover, expression blending plays an important part in Intelligent Human-Computer Interaction, Computer Animation and Comics, entertainment and so on.
The work presented in this paper is as follows.
Firstly, I choose CANDIDE-3 triangle mesh as standard face model. With the aid of graphic toolkit OpenGL’s picking method, I move every vertex of CANDIDE-3 triangle mesh combining with the procedure of 2D facial modeling through Human-Computer Interaction. I get a triangle mesh of the target face model matching his face picture finally.
Secondly, after preprocessing the example face triangle mesh, I suggest a new expression blending algorithm based on the example model’s facial expression. It calculates the expression motion data of sample model and retargets it to the target model’s facial triangle mesh. And then, I adjust the triangle mesh of target face model based on two hypothesizes—every mesh vertex has its own motion zone and the human face is symmetric.
Thirdly, so as to apply textures to target model’s facial triangle mesh, I put forth my image morphing algorithm based on OpenGL texture mapping technology. On the basis of target model’s neutral facial image, its triangle mesh of its neutral facial image, and its another expression triangle mesh, I can get target model’s anther facial expression which is vivid and anti-aliasing.
Finally, reference to every step of my expression blending method I do some experiments step by step to verify the validation of my algorithm.
论文的主要工作和创新如下:?
首先,本文采用CANDIDE‐3模型作为标准的人脸网格模型。通过基于OpenGL的特征点拾取技术的人机交互方式对CANDIDE‐3三角网格模型上顶点的进行调整后,实现了CANDIDE‐3标准模型与特定人脸的匹配,最后得到了一个符合特定人脸特征的三角网格模型。?
其次,对样例人脸的三角网格模型进行倾斜度校正等预处理后,本文实现的基于样例的表情映射算法把样例人脸的表情运动数据,映射到目标人脸网格模型上,计算出目标人脸相应表情的三角网格模型。由于表情映射后网格模型上存在一些奇异点,本文对这个网格模型添加了两个约束,一是对模型上网格点的运动范围进行约束,二是依据人脸的对称性对网格模型上的网格点添加对称性约束。?
第三,基于OpenGL纹理映射技术的图像变形算法,对表情映射生成的三角网格模型进行纹理贴图。本算法依据目标人脸的中性表情图像、中性表情对应的人脸三角网格模型和表情映射后的三角网格模型数据,得到了目标人脸的逼真的不走样的表情图像。?
最后,本文对所提出的表情生成算法的相关步骤逐步进行了一系列实验,最后对整体算法给出了综合实验和交叉验证实验,验证了算法的有效性。?
Example-Based Expression Analogy can be defined as: given an neutral expression image and its another expression image, expression blending retargets the sample's expression motion data to another neutral expression image and gets its another expression image.
My work, example-based expression analogy, mainly including human face modeling, expression transformation, texture mapping, is of significance to purse further study on expression morph. Moreover, expression blending plays an important part in Intelligent Human-Computer Interaction, Computer Animation and Comics, entertainment and so on.
The work presented in this paper is as follows.
Firstly, I choose CANDIDE-3 triangle mesh as standard face model. With the aid of graphic toolkit OpenGL’s picking method, I move every vertex of CANDIDE-3 triangle mesh combining with the procedure of 2D facial modeling through Human-Computer Interaction. I get a triangle mesh of the target face model matching his face picture finally.
Secondly, after preprocessing the example face triangle mesh, I suggest a new expression blending algorithm based on the example model’s facial expression. It calculates the expression motion data of sample model and retargets it to the target model’s facial triangle mesh. And then, I adjust the triangle mesh of target face model based on two hypothesizes—every mesh vertex has its own motion zone and the human face is symmetric.
Thirdly, so as to apply textures to target model’s facial triangle mesh, I put forth my image morphing algorithm based on OpenGL texture mapping technology. On the basis of target model’s neutral facial image, its triangle mesh of its neutral facial image, and its another expression triangle mesh, I can get target model’s anther facial expression which is vivid and anti-aliasing.
Finally, reference to every step of my expression blending method I do some experiments step by step to verify the validation of my algorithm.
引文
[1] Parke F I, Computer generated animation of faces[C] //Proc. ACM annual conference, Vol. 1, 1972:451-457.
[2] Paul Ekman. Facial Action Coding System–Investigator’s Guide, Research Nexus, 1978.
[3] S.M.Platt and N.I.Badler. Animating facial expression. ACM SIGGRAPH Conference Proceedings,15(3):245–252, 1981.
[4] Waters, K. A Muscle Model for Animating Three-Dimensional Facial Expression. Proceedings of 14th Annual Conference on Computer graphics and interactive techniques. 1987.
[5] Kalra P, Mangli A, Thalmann N M, Thalmann D. Simulation of facial muscle actions based on rational freeform deformation. Computer Graphic Forum, 1992, 11(3): 59--69.
[6] LEE Y C, TERZOPOULOS D, WATERS K. Constructing physics-based facial model of individuals. Proc of Graphics Interface. [ S. l. ] : Canadian Information Processing Society, 1993: 128.
[7] L. Williams. Performance-driven facial animation. Computer Graphics (Proc. SIGGRAPH’90), 24(4), 1990, 235-242.
[8] J. Noh, U. Neumann, Expression cloning, in: SIGGRAPH, ACM SIGGRAPH, 2001, pp. 277–288.?
[9]郑飞,陈梅.基于正面头像的人面三维造型.计算机学报, 1998年第11期.
[10]晏洁,高文.基于一般人脸模型修改的特定人脸合成技术.计算机辅助设计与图形学学报, 1999, 11 (05): 389-394.
[11]晏洁,高文,尹宝才.具有真实感的三维虚拟特定人脸生成方法.计算机学报, 1999, 22 (02): 147-153.
[12]梅丽,鲍虎军,彭群生.特定人脸的快速定制和肌肉驱动的表情动画.计算机辅助设计与图形学学报, 2001, 13 (12): 1077-1082.
[13]尹宝才、高文.基于模型的头部运动估计和面部表情合成[J].计算机研究与发展,1999,36(1):67-71.
[14]虞露. MPEG-4中脸部动画参数和序列重绘的肌肉模型,?Journal of Image and Graphic, Vol.6(A), No.1, 2001.1, pp.36-38.
[15]?http://www.icg.isy.liu.se/candide/main.html.
[16]?[美]Rechard S.Wright, Jr Michael Sweet,潇湘工作室译.OpengL SuperBible 3rd(OpenGL超级宝典?第三版).北京:人民邮电出版社.2001.?
[17] Catmull E. A Subdivision Algorithm for Computer Display of Curved Surfaces[D]. Universituy of Utah,Logan,USA’Utah,1974.
[18]?王志良,?孟秀艳编著.?人脸工程学.?北京:机械工业出版社.?2008.?
[19]?李治国,姜大龙,?高文,?王巍.?全自动的基于MPEG‐4的任意拓扑结构人脸动画.计算机辅助设计与图形学学报,2003,July.?
[20]?David F Rogers(美),?石教英,?等译.?计算机图形学的算法[M].机械工业出版社,2002.?
[21] http://17de.com/library/d3d_6im/d3dim6_19.htm.
[22]?Bier E A,Sloan K R.Two—part texture mappings[J].IEEE Computer Graphics and Application,1986,6(9):40～53.
[23]江巨浪,张佑生.一种适用于球面局部区域的纹理映射算法[J].中国图象图形学报,2004,9(9):1112～1116?
[24]顾耀林?周建林.基于MIP-MAP预过滤的区域取样球面纹理映射[J].计算机工程与设计,2007,28(14):3410-3412.
[25]?Jon P Ewins, Marcus D Waller, Martin White. MIP MAP Level Selection for Texture Mapping[J]. IEEE Transaction on Visualization and Computer Graphics. 1998,4(4):317-329
[26]?K.霍金,?D.阿斯特.?OpenGL游戏程序设计[M].?北京:科学出版社.?2006:168~169.
[27]? (美)安吉尔著,段菲译.?OpenGL编程基础(第3版).?北京:清华大学出版社.?2008(3):167~171.?
[28]?T. F. Cootes, G. J. Edwards, and C. J. Taylor, "Active Appearance Models," IEEE Trans. on PAMI, Vol. 23, No. 6, pp. 681-685, June, 2001.
[29]?蒋宗礼著.?人工神经网络导论.?北京‐高等教育出版社.?2001.? ?
[30] LIU Ai-ping, ZHOU Yan, GUAN Xin-pu. Application of Improved Active Shape Model in Face Positioning. Computer Engineering, Vol33, No.18.September 2007.
[31]?胡峰松,林亚平,邹北骥,张茂军.应用于人脸识别的基于Candide‐3特定人脸三维重建[J].湖南大学学报:自然科学版,2008,35(11).?
[32]?孔德慧,郭金铜,尹宝才,王珂.基于MPEG‐4的三维人脸动画系统.北京工业大学学报,2005(02).? ? ?
[33] Jessica Hodgins and Nancy Pollard. Adapting simulated behaviors for new characters. In Turner Whitted, editor, SIGGRAPH 97 Conference Proceedings, pages 153-12, August 1997.
[34] Yin Baocai, Gao Wen. Radial Basis Function Interpolation on Space Mesh [C] // VirtualProceedings of ACM SIGGRAPH’97. 1997: 150.
[35]李旭东,张振跃.人脸表情的形变线性拟合方法.自动化学报Vol.34, No.5.May, 2008.
[36] Pighin f, Hecher J, Lischinski D, Szeliski R, Salesin D H. Synthesizing realistic facial expression from photograghs. In: Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques. New Yock, USA: ACM Press, 1998. 75-84.
[37] Zhang Q, Liu Z, Guo B, Shum H. Geometry-driven photo-realistic facial expression synthesis. In: Proceedings of ACM SIGGRAPH/Eurographics Symposium on Computer Animation. San Diego, USA: ACM Press, 2003. 177-186.
[38] Lin D T, Huang H. Facial expression morphing and animation with local warping methods. In: Proceedings of International Conference on Image Analysis and Processing. Venice, Italy: IEEE Computer Society, 1999. 594-599.
[39] Lee Y C, Terzopoulos D, Waters K. Realistic modeling for facial animation. In: SIGGRAPH Proceedings. Los Angeles, USA: ACM Press, 1995. 55-62.
[40]王进,潘志庚,叶澄清.一种基于轮廓多边形的图像变形算法[J].?计算机研究与发展.?1999(11).?
[41]?江巨浪,张佑生,薛峰,胡敏.?两步纹理映射的改进算法.?系统仿真学报.?2006(05).?
[42]?徐欣欣,袁华,张凌.?利用颜色和纹理特征的图像过滤方法.?华南理工大学学报(自然科学版).?2004(S1).?
[43] Hildebrandt,Klaus,Polthier.Anisotropic filtering of nonlinear surface features[J].Computer Graphics Forum,2004,23(3):39l—400.
[44] McCormack J,Perry R,Farkas K I.Feline:Fast elliptical lines for anisotropic texture mapping[C].SIGGRAPH Computer Graphics.New York,USA:ACM Press/Addison-Wesley Publishing Co, 1999:243·250.
[45]?邱竹生.光栅扫描雷达显示器的几个关键技术.现代电子,1996,(3):30～35?
[46]?张英杰,张铁昌.?纹理映射技术的发展综述[J].?计算机工程与应用.?March,?1994(Z2).? ?
[47]?陈雪云,张振荣,冷松.?基于Haar小波的眼睛定位方法.?计算机工程.?2010(1).?
[48]?金小刚,鲍虎军,彭群生.?计算机动画技术综述.?软件学报[J],?Vol.8,?No.4.?April,?1997.?
[49] COHEN J D, LIN M C, MANOCHA D, et al. I-COLLIDE an interactive and exact collision Detection system for large-scale environments[ C]// Proceedings of ACM Interactive 3D Graphics Conference. New York: ACM Press, 1995:189 - 196.
[50]?王磊,邹北骥,彭小宁,周凌.?一种改进的提取人脸面部特征点的AAM?拟合算法.?电子学报[J],?Vol.34,?No.8.?Aug.?2006.?
[2] Paul Ekman. Facial Action Coding System–Investigator’s Guide, Research Nexus, 1978.
[3] S.M.Platt and N.I.Badler. Animating facial expression. ACM SIGGRAPH Conference Proceedings,15(3):245–252, 1981.
[4] Waters, K. A Muscle Model for Animating Three-Dimensional Facial Expression. Proceedings of 14th Annual Conference on Computer graphics and interactive techniques. 1987.
[5] Kalra P, Mangli A, Thalmann N M, Thalmann D. Simulation of facial muscle actions based on rational freeform deformation. Computer Graphic Forum, 1992, 11(3): 59--69.
[6] LEE Y C, TERZOPOULOS D, WATERS K. Constructing physics-based facial model of individuals. Proc of Graphics Interface. [ S. l. ] : Canadian Information Processing Society, 1993: 128.
[7] L. Williams. Performance-driven facial animation. Computer Graphics (Proc. SIGGRAPH’90), 24(4), 1990, 235-242.
[8] J. Noh, U. Neumann, Expression cloning, in: SIGGRAPH, ACM SIGGRAPH, 2001, pp. 277–288.?
[9]郑飞,陈梅.基于正面头像的人面三维造型.计算机学报, 1998年第11期.
[10]晏洁,高文.基于一般人脸模型修改的特定人脸合成技术.计算机辅助设计与图形学学报, 1999, 11 (05): 389-394.
[11]晏洁,高文,尹宝才.具有真实感的三维虚拟特定人脸生成方法.计算机学报, 1999, 22 (02): 147-153.
[12]梅丽,鲍虎军,彭群生.特定人脸的快速定制和肌肉驱动的表情动画.计算机辅助设计与图形学学报, 2001, 13 (12): 1077-1082.
[13]尹宝才、高文.基于模型的头部运动估计和面部表情合成[J].计算机研究与发展,1999,36(1):67-71.
[14]虞露. MPEG-4中脸部动画参数和序列重绘的肌肉模型,?Journal of Image and Graphic, Vol.6(A), No.1, 2001.1, pp.36-38.
[15]?http://www.icg.isy.liu.se/candide/main.html.
[16]?[美]Rechard S.Wright, Jr Michael Sweet,潇湘工作室译.OpengL SuperBible 3rd(OpenGL超级宝典?第三版).北京:人民邮电出版社.2001.?
[17] Catmull E. A Subdivision Algorithm for Computer Display of Curved Surfaces[D]. Universituy of Utah,Logan,USA’Utah,1974.
[18]?王志良,?孟秀艳编著.?人脸工程学.?北京:机械工业出版社.?2008.?
[19]?李治国,姜大龙,?高文,?王巍.?全自动的基于MPEG‐4的任意拓扑结构人脸动画.计算机辅助设计与图形学学报,2003,July.?
[20]?David F Rogers(美),?石教英,?等译.?计算机图形学的算法[M].机械工业出版社,2002.?
[21] http://17de.com/library/d3d_6im/d3dim6_19.htm.
[22]?Bier E A,Sloan K R.Two—part texture mappings[J].IEEE Computer Graphics and Application,1986,6(9):40～53.
[23]江巨浪,张佑生.一种适用于球面局部区域的纹理映射算法[J].中国图象图形学报,2004,9(9):1112～1116?
[24]顾耀林?周建林.基于MIP-MAP预过滤的区域取样球面纹理映射[J].计算机工程与设计,2007,28(14):3410-3412.
[25]?Jon P Ewins, Marcus D Waller, Martin White. MIP MAP Level Selection for Texture Mapping[J]. IEEE Transaction on Visualization and Computer Graphics. 1998,4(4):317-329
[26]?K.霍金,?D.阿斯特.?OpenGL游戏程序设计[M].?北京:科学出版社.?2006:168~169.
[27]? (美)安吉尔著,段菲译.?OpenGL编程基础(第3版).?北京:清华大学出版社.?2008(3):167~171.?
[28]?T. F. Cootes, G. J. Edwards, and C. J. Taylor, "Active Appearance Models," IEEE Trans. on PAMI, Vol. 23, No. 6, pp. 681-685, June, 2001.
[29]?蒋宗礼著.?人工神经网络导论.?北京‐高等教育出版社.?2001.? ?
[30] LIU Ai-ping, ZHOU Yan, GUAN Xin-pu. Application of Improved Active Shape Model in Face Positioning. Computer Engineering, Vol33, No.18.September 2007.
[31]?胡峰松,林亚平,邹北骥,张茂军.应用于人脸识别的基于Candide‐3特定人脸三维重建[J].湖南大学学报:自然科学版,2008,35(11).?
[32]?孔德慧,郭金铜,尹宝才,王珂.基于MPEG‐4的三维人脸动画系统.北京工业大学学报,2005(02).? ? ?
[33] Jessica Hodgins and Nancy Pollard. Adapting simulated behaviors for new characters. In Turner Whitted, editor, SIGGRAPH 97 Conference Proceedings, pages 153-12, August 1997.
[34] Yin Baocai, Gao Wen. Radial Basis Function Interpolation on Space Mesh [C] // VirtualProceedings of ACM SIGGRAPH’97. 1997: 150.
[35]李旭东,张振跃.人脸表情的形变线性拟合方法.自动化学报Vol.34, No.5.May, 2008.
[36] Pighin f, Hecher J, Lischinski D, Szeliski R, Salesin D H. Synthesizing realistic facial expression from photograghs. In: Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques. New Yock, USA: ACM Press, 1998. 75-84.
[37] Zhang Q, Liu Z, Guo B, Shum H. Geometry-driven photo-realistic facial expression synthesis. In: Proceedings of ACM SIGGRAPH/Eurographics Symposium on Computer Animation. San Diego, USA: ACM Press, 2003. 177-186.
[38] Lin D T, Huang H. Facial expression morphing and animation with local warping methods. In: Proceedings of International Conference on Image Analysis and Processing. Venice, Italy: IEEE Computer Society, 1999. 594-599.
[39] Lee Y C, Terzopoulos D, Waters K. Realistic modeling for facial animation. In: SIGGRAPH Proceedings. Los Angeles, USA: ACM Press, 1995. 55-62.
[40]王进,潘志庚,叶澄清.一种基于轮廓多边形的图像变形算法[J].?计算机研究与发展.?1999(11).?
[41]?江巨浪,张佑生,薛峰,胡敏.?两步纹理映射的改进算法.?系统仿真学报.?2006(05).?
[42]?徐欣欣,袁华,张凌.?利用颜色和纹理特征的图像过滤方法.?华南理工大学学报(自然科学版).?2004(S1).?
[43] Hildebrandt,Klaus,Polthier.Anisotropic filtering of nonlinear surface features[J].Computer Graphics Forum,2004,23(3):39l—400.
[44] McCormack J,Perry R,Farkas K I.Feline:Fast elliptical lines for anisotropic texture mapping[C].SIGGRAPH Computer Graphics.New York,USA:ACM Press/Addison-Wesley Publishing Co, 1999:243·250.
[45]?邱竹生.光栅扫描雷达显示器的几个关键技术.现代电子,1996,(3):30～35?
[46]?张英杰,张铁昌.?纹理映射技术的发展综述[J].?计算机工程与应用.?March,?1994(Z2).? ?
[47]?陈雪云,张振荣,冷松.?基于Haar小波的眼睛定位方法.?计算机工程.?2010(1).?
[48]?金小刚,鲍虎军,彭群生.?计算机动画技术综述.?软件学报[J],?Vol.8,?No.4.?April,?1997.?
[49] COHEN J D, LIN M C, MANOCHA D, et al. I-COLLIDE an interactive and exact collision Detection system for large-scale environments[ C]// Proceedings of ACM Interactive 3D Graphics Conference. New York: ACM Press, 1995:189 - 196.
[50]?王磊,邹北骥,彭小宁,周凌.?一种改进的提取人脸面部特征点的AAM?拟合算法.?电子学报[J],?Vol.34,?No.8.?Aug.?2006.?