基于HMM及三维人脸模型的情感虚拟人研究
|
摘要
人机交互技术是世界各国在21世纪重点研究的一项关键技术,它是当前信息产业竞争的一个焦点。虚拟人合成技术为实现和谐的人机交互提供了重要的理论与技术基础。
本文对情绪状态转移模型和三维人脸建模及表情合成技术进行了探讨和研究,编程实现了基于三维人脸模型的可以进行人机交互的情感虚拟人系统。在建立情感模型之前,首先研究了隐马尔可夫模型的基本思想和基本算法。在熟悉掌握HMM各种算法的前提下,对将要应用于情感建模的前向—后向算法,Viterbi算法,Baum-Welch算法进行了编程实现。在研究了基于HMM的情感模型的前提下,用Baum-Welch算法实现了情感转移概率矩阵和某种心情状态下表情输出概率矢量的参数估计。在情感模型构建的过程中,对刺激信号进行了更细致的划分,构造了情感模型的初始状态转移矩阵,为刺激信号进一步复杂化提供了一种可借鉴的计算方法。在三维人脸建模方面,本文以包含75个节点的Candide模型作为参数化面部模型,采用多边形表示法中的三角形网格的形式,应用参数法在VC++环境下实现了人脸模型。对网格进行纹理贴图后,通过对控制人脸大部分表情合成的AU(Action Units)的控制来合成人脸模型的面部表情,达到参数化控制的目的,进而可以实时和精确地合成多种表情。
实验表明,本课题设计的情感虚拟人系统所模拟得到的情感反应是符合人类情感行为规律的,为实现和谐的人机交互提供了一种可行的设计方法。
Human Computer Interaction (HCI) is a key technology that all the countries put emphasis on in the 21st century, and it is a focus of competition in the information industry. Virtual Human synthesis technology has provided an important basis for the theory and technology in the realization of a harmonious HCI.
In this paper, we study the affective state transition model, 3D face modeling and expression synthesis technologies and realize an affective virtual human system based on the three-dimensional model of the human face by computing. After studying the basic theory and method of HMM, We program the forward-backward arithmetic, Viterbi arithmetic and Baum-Welch arithmetic. We get the parameter of matrix of emotion transfer probability and of the vector of expression output probability. In emotion modeling, we present more detailed stimulatory signal and construct the initial matrix of emotion transfer probability, and it provide a method for complicating the stimulatory signal. As for the carrier of the emotion, we use Candide face model of 75 vertexes as the basic model to fit different 2D images. In the expression synthesis, we construct the face model under the environment of VC++ by using the triangular grid and synthesize the expression by controlling the AUs (Action Units) which can synthesize most of the human expressions. Then we can synthesize many real-time expressions accurately.
The experiment indicates that the emotion feedback simulated in this virtual human system measures up the rule of the human emotion behavior. This method can provide a feasible way to design for the better HCI environment.
本文对情绪状态转移模型和三维人脸建模及表情合成技术进行了探讨和研究,编程实现了基于三维人脸模型的可以进行人机交互的情感虚拟人系统。在建立情感模型之前,首先研究了隐马尔可夫模型的基本思想和基本算法。在熟悉掌握HMM各种算法的前提下,对将要应用于情感建模的前向—后向算法,Viterbi算法,Baum-Welch算法进行了编程实现。在研究了基于HMM的情感模型的前提下,用Baum-Welch算法实现了情感转移概率矩阵和某种心情状态下表情输出概率矢量的参数估计。在情感模型构建的过程中,对刺激信号进行了更细致的划分,构造了情感模型的初始状态转移矩阵,为刺激信号进一步复杂化提供了一种可借鉴的计算方法。在三维人脸建模方面,本文以包含75个节点的Candide模型作为参数化面部模型,采用多边形表示法中的三角形网格的形式,应用参数法在VC++环境下实现了人脸模型。对网格进行纹理贴图后,通过对控制人脸大部分表情合成的AU(Action Units)的控制来合成人脸模型的面部表情,达到参数化控制的目的,进而可以实时和精确地合成多种表情。
实验表明,本课题设计的情感虚拟人系统所模拟得到的情感反应是符合人类情感行为规律的,为实现和谐的人机交互提供了一种可行的设计方法。
Human Computer Interaction (HCI) is a key technology that all the countries put emphasis on in the 21st century, and it is a focus of competition in the information industry. Virtual Human synthesis technology has provided an important basis for the theory and technology in the realization of a harmonious HCI.
In this paper, we study the affective state transition model, 3D face modeling and expression synthesis technologies and realize an affective virtual human system based on the three-dimensional model of the human face by computing. After studying the basic theory and method of HMM, We program the forward-backward arithmetic, Viterbi arithmetic and Baum-Welch arithmetic. We get the parameter of matrix of emotion transfer probability and of the vector of expression output probability. In emotion modeling, we present more detailed stimulatory signal and construct the initial matrix of emotion transfer probability, and it provide a method for complicating the stimulatory signal. As for the carrier of the emotion, we use Candide face model of 75 vertexes as the basic model to fit different 2D images. In the expression synthesis, we construct the face model under the environment of VC++ by using the triangular grid and synthesize the expression by controlling the AUs (Action Units) which can synthesize most of the human expressions. Then we can synthesize many real-time expressions accurately.
The experiment indicates that the emotion feedback simulated in this virtual human system measures up the rule of the human emotion behavior. This method can provide a feasible way to design for the better HCI environment.
引文
[1]张福增.谈人工智能,福建电脑,2003,(7):1-2
[2]宗成庆,计算机能理解自然语言吗--关于人工智能问题的哲学思考,山东工业大学学报,1997,(2):84-86
[3]盛锦华,人工智能研究的进展与发展趋势,安徽大学学报,1996,(1):102-106
[4]王培铎,人工智能的多层涵义,武警学院学报,1999,(2):53-55
[5]R Nakatsu.Emotion Recognition and Its Application to Computer Agents With Spontaneous Interactive Capabilities.Knowledge-Based Systems,2000,(13):20-26
[6]Kimmy S Kee.Perception of Emotion and Neurocognitive Functioning in chizophrenia:What's The Link.Psychiatry Research,1998,(81):33-35
[7]涂序彦,人工生命及应用,中国人工智能学会第一届“人工生命及应用”专题学术会议论文集[C],中国人工智能协会,2002.10,北京
[8]罗森林、潘丽敏,情感计算理论与技术[D],系统工程与电子技术125卷17期2003年
[9]AGVuillieme,TKCapin,ISPandzic,NMT halmann DThalmarm.Non-verbal Communication Interface for Collaborative Virtual Environments[D].The Virtual Reality Journal.Springer.1999.4:49-59
[10]钟义信,机制主义:人工智能的统一理论,电子学报,2006,34(2):317-321
[11]王志良,人工心理学--关于更接近人脑工作模式的科学,北京科技大学学报[J]2000,10:478-481
[12]王志良,人工心理与人工情感.智能系统学报[J],2006,1(1):38-43
[13]林宗楷,数字化虚拟人体应用平台,香山科学会议第174次学术讨论会文集:“中国数字化虚拟人体的科技问题”,2001,北京,pp54-56
[14]Takacs B,Kiss B,The virtual human interface:A photorealistic digital human Computer Graphics and Applications,IEEE,Volume::23,Issue:5,Sept.Oct.2003.
[15]Weiser M.The computer for the 21 st century,Scientific American,1991,265(3):94 104
[16]Georg Glaeser,Hellmuth Stachel.Open Geometry:OpenGL+Advanced Geometry Springer-Verlag,1999,16(8)
[17]A Ortony,G Clore,A Collins.The Cognitive Structure of Emotions.Cambridge University Press,1988
[18]David R Shaffer,Social and Personality Development.Brooks/Cole,1994:185-188
[19]Gomi T,Vardalas J,Koh-Ichi Ide.Elements of Artificial Emotion.Robot and Human Communication,1995:265-268
[20]Kort B,Reilly R,Picard R W.An Affective Model of Interplay between Emotions and Learning.Advanced Learning Technologies,2001:43-46
[21]Picard R W,Vyzas E,Healey J.Toward Machine Emotional Intelligence-Analysis of Affective Physiological State,IEEE Transactions,2001,(23):1175-1191
[22]薛为民、王志良,基于AGENT的人机情感交互系统研究[D],计算机工程与应用,38卷19期,2002年
[23]徐琳、袁保宗、高文,真实感人脸建模研究的进展与展望,软件学报田,2003,14(4):804-810
[24]M.Rydfalk.CANDIDE,a parameterized face[R],Technical Report,IiTH-ISY-1-0866,Linkoping University,1987
[25]Y.Amit,D.Geman,and B.Jedynak Efficient focusing and face detection,in Face Recognition:From Theory to Application[C],Springer-Verlag,BerlinfNewYork,1998
[26]王玉洁,基于人工心理的情感建模及人工情感交互技术研究,北京科技大学博士学位论文,2007年11月
[27]A Raouzaiou,N Tsapatsoulis,K Karpouzis,S Kollias.Parameterized facial expression synthesis based on M PEG-4[J],EURASIP Journal on Applied Signal processing 2002,10-1021-1038
[28]Kin-Man Lam,Hong Yan.An Analytic-to-Holistic Approach for Face Recognition Based on a Single Frontal View[J],IEEE Trans on P AM1,1998,7(20)
[29]马希荣,远程教育中和谐人机情感交互模型的研究,计算机科学,2005年第32卷第9期,2005年9月
[30]Oatley K,Jenkins J.M.Understanding Emotions.Blackwell,1996
[31]http://artificial-emotion.com
[32]魏哲华,基于人工心理理论的情感机器人的情感计算研究[D],北京科技大学硕士学位论文,2002
[33]陆传责,工程系统中的随机过程,北京电子工业出版社,2000.204页
[2]宗成庆,计算机能理解自然语言吗--关于人工智能问题的哲学思考,山东工业大学学报,1997,(2):84-86
[3]盛锦华,人工智能研究的进展与发展趋势,安徽大学学报,1996,(1):102-106
[4]王培铎,人工智能的多层涵义,武警学院学报,1999,(2):53-55
[5]R Nakatsu.Emotion Recognition and Its Application to Computer Agents With Spontaneous Interactive Capabilities.Knowledge-Based Systems,2000,(13):20-26
[6]Kimmy S Kee.Perception of Emotion and Neurocognitive Functioning in chizophrenia:What's The Link.Psychiatry Research,1998,(81):33-35
[7]涂序彦,人工生命及应用,中国人工智能学会第一届“人工生命及应用”专题学术会议论文集[C],中国人工智能协会,2002.10,北京
[8]罗森林、潘丽敏,情感计算理论与技术[D],系统工程与电子技术125卷17期2003年
[9]AGVuillieme,TKCapin,ISPandzic,NMT halmann DThalmarm.Non-verbal Communication Interface for Collaborative Virtual Environments[D].The Virtual Reality Journal.Springer.1999.4:49-59
[10]钟义信,机制主义:人工智能的统一理论,电子学报,2006,34(2):317-321
[11]王志良,人工心理学--关于更接近人脑工作模式的科学,北京科技大学学报[J]2000,10:478-481
[12]王志良,人工心理与人工情感.智能系统学报[J],2006,1(1):38-43
[13]林宗楷,数字化虚拟人体应用平台,香山科学会议第174次学术讨论会文集:“中国数字化虚拟人体的科技问题”,2001,北京,pp54-56
[14]Takacs B,Kiss B,The virtual human interface:A photorealistic digital human Computer Graphics and Applications,IEEE,Volume::23,Issue:5,Sept.Oct.2003.
[15]Weiser M.The computer for the 21 st century,Scientific American,1991,265(3):94 104
[16]Georg Glaeser,Hellmuth Stachel.Open Geometry:OpenGL+Advanced Geometry Springer-Verlag,1999,16(8)
[17]A Ortony,G Clore,A Collins.The Cognitive Structure of Emotions.Cambridge University Press,1988
[18]David R Shaffer,Social and Personality Development.Brooks/Cole,1994:185-188
[19]Gomi T,Vardalas J,Koh-Ichi Ide.Elements of Artificial Emotion.Robot and Human Communication,1995:265-268
[20]Kort B,Reilly R,Picard R W.An Affective Model of Interplay between Emotions and Learning.Advanced Learning Technologies,2001:43-46
[21]Picard R W,Vyzas E,Healey J.Toward Machine Emotional Intelligence-Analysis of Affective Physiological State,IEEE Transactions,2001,(23):1175-1191
[22]薛为民、王志良,基于AGENT的人机情感交互系统研究[D],计算机工程与应用,38卷19期,2002年
[23]徐琳、袁保宗、高文,真实感人脸建模研究的进展与展望,软件学报田,2003,14(4):804-810
[24]M.Rydfalk.CANDIDE,a parameterized face[R],Technical Report,IiTH-ISY-1-0866,Linkoping University,1987
[25]Y.Amit,D.Geman,and B.Jedynak Efficient focusing and face detection,in Face Recognition:From Theory to Application[C],Springer-Verlag,BerlinfNewYork,1998
[26]王玉洁,基于人工心理的情感建模及人工情感交互技术研究,北京科技大学博士学位论文,2007年11月
[27]A Raouzaiou,N Tsapatsoulis,K Karpouzis,S Kollias.Parameterized facial expression synthesis based on M PEG-4[J],EURASIP Journal on Applied Signal processing 2002,10-1021-1038
[28]Kin-Man Lam,Hong Yan.An Analytic-to-Holistic Approach for Face Recognition Based on a Single Frontal View[J],IEEE Trans on P AM1,1998,7(20)
[29]马希荣,远程教育中和谐人机情感交互模型的研究,计算机科学,2005年第32卷第9期,2005年9月
[30]Oatley K,Jenkins J.M.Understanding Emotions.Blackwell,1996
[31]http://artificial-emotion.com
[32]魏哲华,基于人工心理理论的情感机器人的情感计算研究[D],北京科技大学硕士学位论文,2002
[33]陆传责,工程系统中的随机过程,北京电子工业出版社,2000.204页