摘要
情绪是指人对认知内容的特殊态度,是以个体的愿望和需要为中介的一种心理活动。生活中人们有着各种各样的情绪,每一种情绪同时对人们的生活有反作用。因此对情绪进行识别,并相应地对情绪强度进行估计,用以指导人们的生活有着重要的现实意义。本文作者利用计算机技术,针对急躁这一负面情绪进行研究,并且将其扩展到其他情绪,具体工作如下:
由于心理学家认为人的情绪通过面部表情和动作表情体现,其中的动作表情是指与情绪相关的除去面部表情以外的动作,它包括头部和身体其他部分的运动。因此本文作者将面部表情与动作表情(头部运动)二者结合来估计情绪强度。
首先,对急躁情绪进行数学建模,构建了双模态特征模型,分别采用两种模态信息——人脸表情和头部运动信息来表征急躁情绪。其原因是根据实验观察,人在急躁时往往处于厌恶表情且伴有头部频繁地摆动。根据这一特点,设计用表情强度和头部摆动强度来度量情绪。表情强度采用局部特征方法计算;头部摆动强度采用头部摆动的幅度和频率进行估计,其中幅度采用情绪特征数据的方差进行计算,而频率采用情绪特征数据经DCT变换的期望值来计算。该方法能够扩展到其他情绪,为情绪强度估计提供了一种通用的特征提取方法。实验结果表明该方法能较准确的提取情绪特征。
其次,针对急躁情绪强度估计这一问题,为了更好地结合面部表情与头部运动信息,本文作者采用双层隐马尔科夫模型(DMHMM)对情绪强度进行估计。第一层MHMM对面部表情进行识别,进而得到面部表情强度信息;第二层结合面部表情特征与头部运动特征进行情绪强度的估计,得到情绪强度的三个等级。实验结果表明:该算法实现简单,并且能快速的对情绪强度进行估计,识别的正确率超过了只采用面部表情信息的方法。
Emotion is a special attitude at human's cognized content, and a kind of mind which takes individual's will and demand as resonance. There are various emotions in our living and each of them reacts to our living at the same time. As a result, it has a crucial meaning to give lives a guidance that recognizing emotion and estimating its intensity. We research on a negative emotion—impatience as well as other emotion by computer technoledge as follows:
Psychologists consider that emotion is expressed by facial expression and action's expression. Action's expression is defined as these movements that have something to do with emotion except facial expression. Then we estimate the emotion intensity by facial expression combined with action's expression (head movement).
Firstly, we advance a mathematical modeling for impatience to build a double-mode feature model, expressing impatience with both facial and action's expression. By observation, the reason is that disgust expression is always put on people's face with his head waving frequently when he is impatient. According to this, we plan to measure emotion intensity by expression and head waving intensity. The expression intensity is measured by part-feature method. The amplitude of head waving is measured by variance of feature data, while the frequency by expectations concluded from the DCT changes of feature data. And the head waving intensity is estimated by the amplitude and frequence. This method can be used to estimate other emotions, which become a universal feature model. The experiment results indicate that this method can be used to extract emotional feature precisely.
Secondly, we estimate emotion by Double layers of Gaussian Mixture Hidden Markov Model (DMHMM) for the estimation of impatience intensity. DMHMM recognizes the facial expression to capture its intensity in the First Layer and concludes three levels of emotional intensity by estimating the emotional intensities of facial expression's feature and head waving's feature in the Second Layer. The experiment results indicate that it is a simple and effective algorithm of intensity estimation and the accuracy is heightened compared with the method based on facial expression.
引文
[1]罗森林,潘丽敏.情感计算理论与技术[J].系统工程与电子技术,2003,25(7):907-908
[2]刘箴.虚拟人情绪向量和表情向量的合成[J].系统仿真学报,2006.135-138
[3]王满珍.教师情绪与课堂教学效果[J].中国教育研究与创新,2007,7(4):46-48
[4]钟月欣.不良情绪使炎症水平升高[J].心血管病防治知识,2007.27-29
[5]Murphy.Kevin P.An introduction to Bayesian machine learning[M].2006
[6]Russell Stuart,Norvig Peter.Artificial Intelligence:A Modern Approach(2nd Edition)[M].London:Pearson Education,2003
[7]贺前华,陆以勤,韦岗.一种新的HMM训练方法[J].电子学报,2000,28(9):98-102
[8]Gelman.A,Carlin.H,Rubin.D.Bayesian data analysis(2ndedition)[M].Chapman and Hall,2004
[9]VI.Pavlovic.Dynamic Bayesian Networks for Information Fusion with Applications to Human-Computer Interfaces[D].Univ.of Illinois at Urbana-Champaign,1999
[10]杨国亮,王志良,任金霞.采用Adaboost算法进行面部表情识别[J].计算机应用,2005,25(4):65-68
[11]Wang Yubo,Ai Haizhou,Wu Bo,et al.Real Time Facial Expression Recognition with Adaboost[C].In:Proceedings of the 17th International Conference on Pattern Recognition,2004.1021-1025
[12]丹尼尔.高曼.情商[M].北京:时报文化,1996
[13]詹姆士.什么是情绪[J].心理,1884.35-39
[14]Ekman.P.Facial Expression of Emotion:An Old Controversy and New Findings[J].Philosophical Transactions:Biological Sciences,1992.680-684
[15]Minsky M.The Society of Mind[M].New York:Simon Schuster,1985
[16]Pizard.RW.Perceptual Computing Technical Report No.321 Affective computing[R].Cambridge:MITMedia Lab,1995
[17]Inokuchi Seiji,Kansei.Information Processing[J].Journal of Japan Society for Institute of Elecrronics Information and Communicatiom Engineers,1997,80(10):1007-1012
[18]RW.Picard.Afective Computing:Challenges[J].International Joumal of Human Computer Studies,2003,59(6):55-64
[19]胡包钢,谭铁牛,王珏等.情感计算—计算机科技发展的新课题[N].科学时报,2002
[20]万军红.情感计算[J].上海机电学院学报,2007,4(10):300-304
[21]B.Fasel,J.Luettin.Automatic facial expression analysis:a survey[J].Pattern Recognition,2003,36(1):259-275
[22]Suwa.M,Sugie.N,Fujimora.K.A Preliminary Note on Pattem Recognition of Human Emotional Expression[C].Kyoto,Japan:Proceedings of the 4th International Joint Conference on Pattern Recognition,1978.408-410
[23]Ekman.P,Friesen.WV.Facial action coding system:A technique for the measurement of facial movement[C].Palo Alto,Calif:Consulting Psychologists Press,1978.325-330
[24]何良华,邹彩荣,包永强等.人脸面部表情识别的研究进展[J].电路与系统报,2005,10(1):70-75
[25]Cootes Timothy F,Edwards Gareth J,et al.Active Appearance Models[J].IEEE Transactions on Pami,2001,23(6):102-106
[26]余冰等.基于特征运动的表情人脸识别[J].中国图形图像学报,2002,7(11):425-430
[27]Michael J.Lyons,Julien Budynek,Shigeru Akamatsu.Automatic Classification of Single Facial Images[J].IEEE Trancations on Pattern Analysis and Machine Intelligence,1999,21(12):1357-1362
[28]Wang Yubo,Ai Haizhou,Huang Chang.Real Time Facial Expression Recognition with Adaboost[C].International Conferenceon Pattern Recognition,2004.1024-1028
[29]Jesse Hoey,James J.Little.Bayesian Clustering of Optical Flow Fields[C].The Ninth IEEE International Conference on Computer Vision,2003.526-530
[30]金辉,高文.基于特征流的面部表情运动分析及应用[J].软件学报,2003,14(12):2098-2105
[31]Yongrnian Zhang,Qiang Ji.Active and Dynamic Information Fusion for Facial ExpressionUnderstanding from Image Sequences[J].IEEE Transactions On Pattern Analysis And Machine Intelligence,2005,27(05):699-714
[32]Ira Cohen,Nicu Sebe,Fabio G.Cozman,et al.Learning Bayesian Network Classifiers for Facial Expression Recognition using both Labeled and Unlabeled Data[C].In:Computer Society Conference on Computer Vision and Pattern Recognition,2003,25(6):752-756
[33]Lawrence.R,Rabiner.A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition[J].Proceedings of the IEEE,1989,77(02):102-106
[34]金辉,高文.基于HMM的面部表情图像序列的分析与识别[J].自动化学报,2002,28(04):647-650
[35]Zhou Xiaoxu,Huang Xiangsheng,Xu Bin,et al.Real-time Facial Expression Recognition Based on Boosted Embedded HiddenMarkov Model[C].In:Third International Conference on Image and Graphics,2004.623-626
[36]Zhou Xiaoxu,Huang Xiangsheng,Xu Bin,et al.Real-time Facial Expression Recognition in the Interactive Game Based on Embedded Hidden Markov Model[C].In:International Conference on Computer Graphics,Imaging and Visualization,2004.225-230
[37]汤丽君,邹北骥,王磊,等.基于MHMMs模型的面部表情识别研究[C].第六届中国计算机图形学大会论文集,2006.112-117
[38]Hu Angbo,Chang Ya,Feris Rogerio,et al.Manifold Based Analysis of Facial Expression[C].In:Computer Society Conference on Computer Vision and Pattern Recognition Workshops,2004.312-318
[39]Hu Angbo,Chang Ya,Feris Rogerio,et al.Probabilistic Expression Analysis on Manifolds[C].In:Computer Society Conference on Computer Vision and Pattern Recognition,2004.215-220
[40]Tenenbaum.JB,Silva.V,Langford.JC.A Global Geometric Framework for Nonlinear Dimensionality Reduction[J].Science,2003.2319-2323
[41]邹北骥,李燕,孙家广,等.基于特征点跟踪的面部表情强度度量[C].第六届中国计算机图形学大会论文集,2006.412-416
[42]Sebe Nicu,S.Lew Michael,Cohen Ira,et al.Authentic Facial Expression Analysis[C].In:the Sixth IEEE International Conference on Automatic Face and Gesture Recognition,2004.825-831
[43]Zhen Wen,Thomas S.Huang.Capturing Subtle Facial Motions in 3D Face Tracking[C].In:the Ninth IEEE International Conference on Computer Vision,2003.1021-1026
[44] Marian Stewart Bartlett, Gwen Littlewort, Mark Frank, et al. Recognizing Facial Expression: Machine Learning and Application to Spontaneous Behavior[C]. In: Computer Society Conference on Computer Vision and Pattern Recognition, 2005.123-126
[45] Su Congyong, Huang Li. Facial Expression Hallucination[J]. The Seventh IEEE Workshop on Applications of Computer Vision, 2005.721-726
[46] Gu Haisong, Ji Qiang. Facial Event Classification with Task Oriented Dynamic Bayesian Network[C]. In: Computer Society Conference on Computer Vision and PatternRecognition, 2004.1063-1069
[47] Gu Haisong, Ji Qiang. An Automated Face Reader for Fatigue Detection[C]. In: Conference on Automatic Face and Gesture Recognition, 2004.1003-1006
[48] Kanade T, Cohn JF, Tian YL. Comprehensive database for facial expression analysis[C]. In: Proc Of the 4th IEEE International Conference on Automatic Face and Gesture Recognition. Grenoble, France: IEEE Computer Society, 2000.46-53
[49] Sim T, Baker S, Bsat M. The CMU pose, illumination and expression(PIE) database[C]. Washington, DC, USA: Proc Of the 5th IEEE International Conference on Automatic Face and Gesture Recognition, 2002.53-58
[50] 吴丹,林学訚。人脸表情视频数据库的设计与实现[J)．计算机工程与应用,2004,40(5)：177-180
[51] Hess.U, Blairy.S, Kleck.R. The Intensity of Emotional Facail Expressions and Decoding Accuracy [J]. Journal of Nonverbal Behavior, 1997, 21(4): 241-257
[52] Matsumoto.D, Kasri.F, Kooken.K. American-Japanese Cultural Differences in Judgements of Expression Intensity and Subjective Experience[J]. Cognition and Emotion, 1999,13(2): 201-218
[53] P.M.Niedenthal, J.B.Halberstadt, J.Margolin, et al. Emotional State and the Detection of Change in Facial Expression of Emotion[J]. European Journal of Social Psychology, 2000.211-222
[54] H.Kobayashi, F.Hara. Recognition of six basic facial expressions and their strength by neural network[C]. In: IEEE International Workshop on Robot and Human Communication, 1992.381-386
[55] S.Kimura, M.Yachida. Facial expression recognition and its degree estimation[C]. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, 1997.295-300
[56]J.J.-J.Lien,T.Kanade,J.Cohn,et al.Subtly different facial expression recognition and expression intensity estimation[C].In:Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition,1998.853-859
[57]Wang Mei,Iwai Yoshio,Masahiko Yachida.Recognizing degree of continuous facial expression change[C].In:Proceedings of the 14th International Conference on Pattern Recognition,1998.1188-1190
[58]Lee Ka Keung,Xu Yangsheng.Real-time estimation of facial expression intensity[C].In:Proceedings of the IEEE International Conference on Robotics & Automations,2003.2567-2572
[59]Kaliouby Rana El,Robinson Peter.Real-Time Inference of Complex Mental Statesfrom Facial Expressions and Head Gestures[C].In:Computer Society Conference on Computer Vision and Pattern Recognition Workshops,2004.63-69
[60]王磊.人脸表情自动提取与跟踪技术研究[D].长沙:湖南大学研究生院,2007
[61]Tibshirani.R,Wakher.G,Hastie.T.Estimating the number of clusters in a dataset via the gapstatistic[J].Royal Statist Sac.B,2001.102-108