基于脉搏信号的情感识别研究
|
摘要
情感识别不仅是人工智能的基础研究领域,也是人机交互领域的一个热门话题。基于生理信号的情感识别比基于面部表情和基于语音的情感识别在实现上更为困难和复杂,但是因为生理信号由人体白发产生,不受人的主观意识的控制,因此在研究过程中更加准确和可靠。实验室建立了规模较大的情感生理信号数据库,主要研究高兴、惊奇、厌恶、悲伤、愤怒和恐惧6种情感状态下由生理过程自发产生的7种生理信号,其中包括电生理信号如心电(ECG)、脑电(EEG)、肌电(EMG)、皮肤电导(GSR)和非电生理信号如心率(Heart Rate)、脉搏(Pulse)、呼吸(RSP)。
脉搏信号一直以来都是一个重要的研究课题,但是基于脉搏信号的情感识别的先行研究还较少,论文主要以脉搏信号,辅以心电信号(Electrocardiography, ECG)为研究对象,通过生理信号处理方法,对这些信号进行滤波和重构,并提取重要特征,同时将局部搜索和变异策略引入到最大最小蚂蚁系统(Max-Min Ant System, MMAS)中,结合Fisher分类器对情感进行识别,获得了较好的效果。本文主要包括了以下工作：
1.对信号进行预处理：利用小波包变换去除信号的基线漂移和工频干扰,用五点三次法对信号进行平滑。
2.通过小波变换准确定位脉搏波的主波波峰,用高斯函数钟形波拟合脉搏波的主波、潮波和重搏波,进而提取特征。
3.因为提取的特征数较多,因此我们使用智能算法进行特征选择。为了改善最大最小蚂蚁系统收敛速度慢,易陷入局部最优解的缺点,将伪随机比例、局部搜索和变异策略引入最大最小蚂蚁系统中,得到改进的最大最小蚂蚁系统(IMMAS)。
4.首先使用104个统计特征作为原始特征集,采用IMMAS算法与Fisher分类器相结合的方法,分别对情感进行“一对一”和“一对多”识别；继而将IMMAS选出的最优统计特征子集与新特征相结合,重新对高兴、惊奇、厌恶、悲伤、愤怒、恐惧进行识别,获得了更好的识别效果。
实验和仿真结果表明,使用脉搏信号进行情感识别是可行的。
Emotion Recognition is not only the basis of artificial intelligence, but also a hot research issue in human-computer interaction (HCI). Emotion recognition based on physiological signals is more difficult and complicated to achieve than that based on facial expressions and speech; however, because physiological signals are produced by human body spontaneously, without conscious control of the subject, so the research process is more accurate and reliable. A fairly large physiological signal database is created by our lab and 7 kinds of physiological signals, including electrophysiological signals such as electrocardiogram (ECG), electroencephalogram (EEG), electroneuromyography (EMG), Galvanic Skin Response (GSR) and non-electrophysiological signals such as heart rate, pulse signal, respiration (RSP), under 6 emotion states, namely, happiness, surprise, disgust, grief, anger and fear.
For quite a long time, pulse signal has been an important research issue; however, there are few prior research of emotion recognition based on pulse signal. Pulse signal aided with ECG signal are studied in this paper, and important features are extracted from the signals after filtering and reconstruction. The Improved Max-Min Ant System (IMMAS), combined with Fisher classifier is used for emotion recognition, and good effect is obtained.
The following work is discussed in this paper:
1. Signal preprocessing:wavelet packet transform is used to remove baseline shift and power-line interference of the signal and five-spot triple method is adopted for signal smoothing.
2. Percussion wave crest of pulse signal is positioned accurately through wavelet transform, and then three Gaussian functions are used to fit the percussion wave, tidal wave and dicrotic wave for feature extraction.
3. Because there are a large number of original features, so intelligent algorithm is used for feature selection. In order to overcome the disadvantages of MMAS, such as slow rate of convergence and liable to trap in local optimum, Pseudo-random proportion, local search and variance strategy are introduced to MMAS and the improved MMAS (IMMAS) is obtained.
4. Firstly, the 104 statistical features are used as the original feature subset, and IMMAS combined with Fisher classifier is adopted one-vs-one emotion recognition and one-vs-rest emotion recognition; then the optimal feature subsets selected by IMMAS combined with some new features are used to recognize happiness, surprise, disgust, grief, anger and fear once again, and better recognition rates are obtained.
The experiment and simulation results show that it is feasible to use pulse signal for feature selection.
脉搏信号一直以来都是一个重要的研究课题,但是基于脉搏信号的情感识别的先行研究还较少,论文主要以脉搏信号,辅以心电信号(Electrocardiography, ECG)为研究对象,通过生理信号处理方法,对这些信号进行滤波和重构,并提取重要特征,同时将局部搜索和变异策略引入到最大最小蚂蚁系统(Max-Min Ant System, MMAS)中,结合Fisher分类器对情感进行识别,获得了较好的效果。本文主要包括了以下工作：
1.对信号进行预处理：利用小波包变换去除信号的基线漂移和工频干扰,用五点三次法对信号进行平滑。
2.通过小波变换准确定位脉搏波的主波波峰,用高斯函数钟形波拟合脉搏波的主波、潮波和重搏波,进而提取特征。
3.因为提取的特征数较多,因此我们使用智能算法进行特征选择。为了改善最大最小蚂蚁系统收敛速度慢,易陷入局部最优解的缺点,将伪随机比例、局部搜索和变异策略引入最大最小蚂蚁系统中,得到改进的最大最小蚂蚁系统(IMMAS)。
4.首先使用104个统计特征作为原始特征集,采用IMMAS算法与Fisher分类器相结合的方法,分别对情感进行“一对一”和“一对多”识别；继而将IMMAS选出的最优统计特征子集与新特征相结合,重新对高兴、惊奇、厌恶、悲伤、愤怒、恐惧进行识别,获得了更好的识别效果。
实验和仿真结果表明,使用脉搏信号进行情感识别是可行的。
Emotion Recognition is not only the basis of artificial intelligence, but also a hot research issue in human-computer interaction (HCI). Emotion recognition based on physiological signals is more difficult and complicated to achieve than that based on facial expressions and speech; however, because physiological signals are produced by human body spontaneously, without conscious control of the subject, so the research process is more accurate and reliable. A fairly large physiological signal database is created by our lab and 7 kinds of physiological signals, including electrophysiological signals such as electrocardiogram (ECG), electroencephalogram (EEG), electroneuromyography (EMG), Galvanic Skin Response (GSR) and non-electrophysiological signals such as heart rate, pulse signal, respiration (RSP), under 6 emotion states, namely, happiness, surprise, disgust, grief, anger and fear.
For quite a long time, pulse signal has been an important research issue; however, there are few prior research of emotion recognition based on pulse signal. Pulse signal aided with ECG signal are studied in this paper, and important features are extracted from the signals after filtering and reconstruction. The Improved Max-Min Ant System (IMMAS), combined with Fisher classifier is used for emotion recognition, and good effect is obtained.
The following work is discussed in this paper:
1. Signal preprocessing:wavelet packet transform is used to remove baseline shift and power-line interference of the signal and five-spot triple method is adopted for signal smoothing.
2. Percussion wave crest of pulse signal is positioned accurately through wavelet transform, and then three Gaussian functions are used to fit the percussion wave, tidal wave and dicrotic wave for feature extraction.
3. Because there are a large number of original features, so intelligent algorithm is used for feature selection. In order to overcome the disadvantages of MMAS, such as slow rate of convergence and liable to trap in local optimum, Pseudo-random proportion, local search and variance strategy are introduced to MMAS and the improved MMAS (IMMAS) is obtained.
4. Firstly, the 104 statistical features are used as the original feature subset, and IMMAS combined with Fisher classifier is adopted one-vs-one emotion recognition and one-vs-rest emotion recognition; then the optimal feature subsets selected by IMMAS combined with some new features are used to recognize happiness, surprise, disgust, grief, anger and fear once again, and better recognition rates are obtained.
The experiment and simulation results show that it is feasible to use pulse signal for feature selection.
引文
[1]John L. Semmlow. Biosignal and Biomedical Image Processing.Marcel Dekker,Inc., New York,2005.
[2]Minsky M..The Society of Mind. Simon & Schuster, New York, NY,1985.
[3]Picard, R. W.. Affective Computing.MIT Press, London, England,1997.
[4]LeDoux, J.E..The Emotional Brain:the Mysterious Underpinning of Emotional Life. Simon&Schuster, NewYork,1996.
[5]Damasio,Antonio.Descartes'Error.Emotion,Reason, and the Human Brain. Picador London,1995.
[6]日本文部省重点研究领域：情感信息处理的信息学、心理学研究,成果报告书,1999.
[7]仇德辉.姗姗而来的情感机器人.科技视野,2009年8月：P25-27.
[8]D. Canamero. "Modeling Motivations and Emotions as a Basis for Intelligent Behavior".Proc. of the First International Symposium on Autonomous Agents,1997.
[9]金辉,高文.人脸面部混合表情识别系统.计算机学报.2000年6月,第23卷,第6期.
[10]宋亦旭,贾培发.基于人工情感的泥人机器人控制体系结构.2004年11月,第26卷第6期.
[11]王延江,袁保宗.多模态人机交互中基于笔输入的手势识别.北方交通大学学报.2001年4月,第25卷,第2期.
[12]陶建华,谭铁牛.语音和人脸表情同步的双模态情感表达研究.第一届中国情感计算及智能交互学术会议论文集.北京：西郊宾馆.2003年12月,P248-253.
[13]王上飞,陈恩红,李金龙等.基于内容的交互式感性图像检索.中国图像图形学报.2001年10月,第6卷(A版),第10期.
[14]周建武,戴国忠.用户界面评估系统UIEV_Pro的设计与实现.1998年3月,第10卷,第2期：P97-104.
[15]朱朝辉,潘志庚,R-Teatrix中虚拟人物及情感系统构造.第一届中国情感计算及智能交互学术会议论文集.北京：西郊宾馆.2003年12月7-8日,P242-247.
[16]D. M. Clynes, Sentics:The Touch of the Emotions. Anchor Press/Doubleday,1977.
[17]Don P.Morris, M.D..Blood Pressure and Pulse Changes in Normal Individuals under Emotional Stress; Their Relationship to Emotional Instability.Phychosomatic Medicine, Vol.3, No.4, October,1941.
[18]Chuan-Yu Chang, Jun-Ying Zheng, Chi-Jane Wang.Based on Support Vector Regression for Emotion Recognition using Physiological Signals.Proceesings of the 2010 International Joint Conference on Neural Networks,July 18-23,2010 P1-7.
[19]C.Maaoui, A.Pruski. A Comparative Study of SVM Kernel Applied to Emotion Recognition from Physiological Signals. the 5th International Multi-Conference On Systems, Signals and Devices,2008.
[20]http://airlab.elet.polimi.it/index.php/Wireless_Affective_Devices.
[21]O.Villon and C.Lisetti, Toward Recognizing Individual's Subjective Emotion from Physiological Signals in Practical Application, Proceedings of 20th IEEE International Symposium on Computer-Based Medical Systems (CBMS2007), IEEE Computer Society Press, Maribor, Slovenia,2007, pp.357-362.
[22]陈曾,刘光远.脑电信号在情感识别中的研究.计算机工程.201036(9)：168-170.
[23]Ying Cheng, Guangyuan Liu, Huiling Zhang.The Research of EMG Signal in Emotion Recognition Based on TS and SBS Algorithm. Proceedings of 2010 the 3rd International Conference on Information Sciences and Interaction Sciences,2010 pp:363-366.
[24]林时来,刘光远,张慧玲.“蚁群算法在呼吸信号情感识别中的应用研究”.计算机工程与应用,2011,47(2)：69-72.
[25]Changwei Ma, Guangyuan Liu et.al.Application of Max-Min Ant System for Emotion Recognition from Electrocardiogram signals.International Journal of Pattern Recognition and Artificial Intelligence,July 16,2009.
[26]郭计云,王福明.基于小波包变换的信号去噪方法研究.现代电子技术,2007年,第19期：P55-56.
[27]张坤,曹鸣.一种基于小波变换的心电去噪算法.现代生物医学进展,2009年,第9卷,第19期：P3744-3746.
[28]葛臣,刘光远,龙正吉.情感识别中脉搏信号的特征提取与分析.西南师范大学学报(自然科学版),2010年6月,第35卷,第3期.
[29]Huiling Zhang, Guangyuan Liu. "Research of Emotion Recognition Based on Pulse Signal".Proceedings of the 2010 the 3rd International Conference on Advanced Computer Theory and Engineering,2010 (03) pp:506-509.
[30]钱伟力.高斯函数分析法提取脉搏波特征.中国生物医学工程学报,1994,13(1)：1-7.
[31]罗志昌,张松,杨益民.脉搏波的工成分析与临床应用.北京：科学出版社,2006.
[32]费兆馥.现代中医脉诊学.北京：人民卫生出版社,2003：1672168.
[33]杨军良.脉搏传递时间(PTT)——麻醉期间一种新的检测手段.中华麻醉在线,2007年9月.
[34]李士勇.蚁群算法及其应用.哈尔滨：哈尔滨工业大学出版社,2004.
[35]Marco Dorigo, Thomas Stutzle(著),张军等(译).蚁群算法.北京：清华大学出版社,2007年1月.
[36]李士勇.蚁群算法及其应用.哈尔滨：哈尔滨工业大学出版社,2004.
[37]陈烨.带杂交算子的蚁群算法.计算机工程.2001,27(12)：74-76.
[38]陈星宇,肖伟,全惠云.应用LK算法求解旅行商问题的混合蚂蚁算法.计算机工程,2008,34(4)：228-231.
[39]吴庆洪,张纪会,徐心和.具有变异特性的蚁群算法.计算机研究与发展,1999,36(10)：1240-1245.
[40]唐增明,蒋泰.一种改进的动态自适应最大-最小蚁群算法.计算机与现代化,2008,3：89-93.
[41]赵军阳,张志利.基于模糊粗糙集信息熵的蚁群特征选择方法计算机应用,2009年1月,第29卷,第1期：P109-126.
[42]杨淑莹,李兰友.图像模式识别.北京：清华大学出版社,北京交通大学出版社.
[43]葛臣.脉搏信号在情感状态识别中的研究：[硕士学位论文].重庆：西南大学,2009年6月.
[2]Minsky M..The Society of Mind. Simon & Schuster, New York, NY,1985.
[3]Picard, R. W.. Affective Computing.MIT Press, London, England,1997.
[4]LeDoux, J.E..The Emotional Brain:the Mysterious Underpinning of Emotional Life. Simon&Schuster, NewYork,1996.
[5]Damasio,Antonio.Descartes'Error.Emotion,Reason, and the Human Brain. Picador London,1995.
[6]日本文部省重点研究领域：情感信息处理的信息学、心理学研究,成果报告书,1999.
[7]仇德辉.姗姗而来的情感机器人.科技视野,2009年8月：P25-27.
[8]D. Canamero. "Modeling Motivations and Emotions as a Basis for Intelligent Behavior".Proc. of the First International Symposium on Autonomous Agents,1997.
[9]金辉,高文.人脸面部混合表情识别系统.计算机学报.2000年6月,第23卷,第6期.
[10]宋亦旭,贾培发.基于人工情感的泥人机器人控制体系结构.2004年11月,第26卷第6期.
[11]王延江,袁保宗.多模态人机交互中基于笔输入的手势识别.北方交通大学学报.2001年4月,第25卷,第2期.
[12]陶建华,谭铁牛.语音和人脸表情同步的双模态情感表达研究.第一届中国情感计算及智能交互学术会议论文集.北京：西郊宾馆.2003年12月,P248-253.
[13]王上飞,陈恩红,李金龙等.基于内容的交互式感性图像检索.中国图像图形学报.2001年10月,第6卷(A版),第10期.
[14]周建武,戴国忠.用户界面评估系统UIEV_Pro的设计与实现.1998年3月,第10卷,第2期：P97-104.
[15]朱朝辉,潘志庚,R-Teatrix中虚拟人物及情感系统构造.第一届中国情感计算及智能交互学术会议论文集.北京：西郊宾馆.2003年12月7-8日,P242-247.
[16]D. M. Clynes, Sentics:The Touch of the Emotions. Anchor Press/Doubleday,1977.
[17]Don P.Morris, M.D..Blood Pressure and Pulse Changes in Normal Individuals under Emotional Stress; Their Relationship to Emotional Instability.Phychosomatic Medicine, Vol.3, No.4, October,1941.
[18]Chuan-Yu Chang, Jun-Ying Zheng, Chi-Jane Wang.Based on Support Vector Regression for Emotion Recognition using Physiological Signals.Proceesings of the 2010 International Joint Conference on Neural Networks,July 18-23,2010 P1-7.
[19]C.Maaoui, A.Pruski. A Comparative Study of SVM Kernel Applied to Emotion Recognition from Physiological Signals. the 5th International Multi-Conference On Systems, Signals and Devices,2008.
[20]http://airlab.elet.polimi.it/index.php/Wireless_Affective_Devices.
[21]O.Villon and C.Lisetti, Toward Recognizing Individual's Subjective Emotion from Physiological Signals in Practical Application, Proceedings of 20th IEEE International Symposium on Computer-Based Medical Systems (CBMS2007), IEEE Computer Society Press, Maribor, Slovenia,2007, pp.357-362.
[22]陈曾,刘光远.脑电信号在情感识别中的研究.计算机工程.201036(9)：168-170.
[23]Ying Cheng, Guangyuan Liu, Huiling Zhang.The Research of EMG Signal in Emotion Recognition Based on TS and SBS Algorithm. Proceedings of 2010 the 3rd International Conference on Information Sciences and Interaction Sciences,2010 pp:363-366.
[24]林时来,刘光远,张慧玲.“蚁群算法在呼吸信号情感识别中的应用研究”.计算机工程与应用,2011,47(2)：69-72.
[25]Changwei Ma, Guangyuan Liu et.al.Application of Max-Min Ant System for Emotion Recognition from Electrocardiogram signals.International Journal of Pattern Recognition and Artificial Intelligence,July 16,2009.
[26]郭计云,王福明.基于小波包变换的信号去噪方法研究.现代电子技术,2007年,第19期：P55-56.
[27]张坤,曹鸣.一种基于小波变换的心电去噪算法.现代生物医学进展,2009年,第9卷,第19期：P3744-3746.
[28]葛臣,刘光远,龙正吉.情感识别中脉搏信号的特征提取与分析.西南师范大学学报(自然科学版),2010年6月,第35卷,第3期.
[29]Huiling Zhang, Guangyuan Liu. "Research of Emotion Recognition Based on Pulse Signal".Proceedings of the 2010 the 3rd International Conference on Advanced Computer Theory and Engineering,2010 (03) pp:506-509.
[30]钱伟力.高斯函数分析法提取脉搏波特征.中国生物医学工程学报,1994,13(1)：1-7.
[31]罗志昌,张松,杨益民.脉搏波的工成分析与临床应用.北京：科学出版社,2006.
[32]费兆馥.现代中医脉诊学.北京：人民卫生出版社,2003：1672168.
[33]杨军良.脉搏传递时间(PTT)——麻醉期间一种新的检测手段.中华麻醉在线,2007年9月.
[34]李士勇.蚁群算法及其应用.哈尔滨：哈尔滨工业大学出版社,2004.
[35]Marco Dorigo, Thomas Stutzle(著),张军等(译).蚁群算法.北京：清华大学出版社,2007年1月.
[36]李士勇.蚁群算法及其应用.哈尔滨：哈尔滨工业大学出版社,2004.
[37]陈烨.带杂交算子的蚁群算法.计算机工程.2001,27(12)：74-76.
[38]陈星宇,肖伟,全惠云.应用LK算法求解旅行商问题的混合蚂蚁算法.计算机工程,2008,34(4)：228-231.
[39]吴庆洪,张纪会,徐心和.具有变异特性的蚁群算法.计算机研究与发展,1999,36(10)：1240-1245.
[40]唐增明,蒋泰.一种改进的动态自适应最大-最小蚁群算法.计算机与现代化,2008,3：89-93.
[41]赵军阳,张志利.基于模糊粗糙集信息熵的蚁群特征选择方法计算机应用,2009年1月,第29卷,第1期：P109-126.
[42]杨淑莹,李兰友.图像模式识别.北京：清华大学出版社,北京交通大学出版社.
[43]葛臣.脉搏信号在情感状态识别中的研究：[硕士学位论文].重庆：西南大学,2009年6月.