基于眼动和脑电技术的机器人情绪行为对用户交互情感的影响研究
|
摘要
目的探索人-机器人交互过程中用户情感的评价方法,揭示机器人情绪行为对用户交互情感的影响。方法设计音乐交互过程(正性音乐、负性音乐)和机器人情绪行为(正性情绪行为、中性情绪行为、负性情绪行为),借助主观评价、眼动追踪和脑电技术,采集19名被试在不同音乐交互过程中的主观评价指标、眼动指标和脑电指标。结果正性音乐交互过程中,机器人正性情绪行为引起的效价、唤醒度、瞳孔直径、额中线theta波相对功率、额区alpha波不对称指数显著大于中性情绪行为;而负性音乐交互过程中,机器人负性情绪行为引起的唤醒度和瞳孔直径显著大于中性情绪行为,效价和额区alpha波不对称指数显著小于中性情绪行为。结论各评价指标具有一致的结果,各指标相互结合能够有效地评价机器人音乐交互过程中用户的情感状态。评价结果表明机器人正性情绪行为(负性情绪行为)对音乐交互过程中用户正性情感(负性情感)具有增强作用,可用于进一步加强机器人音乐交互过程中用户的情感。
Objective The purpose of this study was to find out an effective evaluation method for users' emotions during human-robot interaction process and understand the effect of robotic behaviors on users' interactive emotions.Methods n this study,two musical interaction process including positive music and negative music and three robotic emotional behaviors expressing positive,neutral,and negative emotions were designed. Then,19 participants randomly completed the musical interaction with the robot that exhibited different emotional behaviors. The subjective assessment of emotion,eye-tracking metrics,and electroencephalography signals were collected. Results The results showed that in positive musical interaction,robot's positive behavior significantly enhanced users' valence,arousal,pupil diameter,theta relative power in frontal-middle line,and asymmetry score of frontal alpha,compared with robot's neutral behavior. While in negative musical interaction,robot's negative behavior significantly enhanced users ' arousal and pupil diameter and weaken valence and asymmetry score of frontal alpha,compared with robot's neutral behavior.Conclusion The results from different indicators achieved a consistency and could be conjunctively used to effectively evaluate users' emotions in interaction with a robot. After the evaluation,it was revealed that robot's positive behavior and negative behavior could significantly enhance users ' positive emotion or negative emotion respectively in musical interaction with robot,which can be used to improve users' emotion.
Objective The purpose of this study was to find out an effective evaluation method for users' emotions during human-robot interaction process and understand the effect of robotic behaviors on users' interactive emotions.Methods n this study,two musical interaction process including positive music and negative music and three robotic emotional behaviors expressing positive,neutral,and negative emotions were designed. Then,19 participants randomly completed the musical interaction with the robot that exhibited different emotional behaviors. The subjective assessment of emotion,eye-tracking metrics,and electroencephalography signals were collected. Results The results showed that in positive musical interaction,robot's positive behavior significantly enhanced users' valence,arousal,pupil diameter,theta relative power in frontal-middle line,and asymmetry score of frontal alpha,compared with robot's neutral behavior. While in negative musical interaction,robot's negative behavior significantly enhanced users ' arousal and pupil diameter and weaken valence and asymmetry score of frontal alpha,compared with robot's neutral behavior.Conclusion The results from different indicators achieved a consistency and could be conjunctively used to effectively evaluate users' emotions in interaction with a robot. After the evaluation,it was revealed that robot's positive behavior and negative behavior could significantly enhance users ' positive emotion or negative emotion respectively in musical interaction with robot,which can be used to improve users' emotion.
引文
[1]Tung F W.Child Perception of Humanoid Robot Appearance and Behavior[J].International Journal of Human-Computer Interaction,2016,32(6):493-502.
[2]Breazeal C.Emotion and Sociable Humanoid Robots[J].International Journal of Human-Computer Studies,2003,59(1-2):119-155.
[3]Khosla R,Nguyen K,Chu M T.Human Robot Engagement and Acceptability in Residential Aged Care[J].International Journal of Human-Computer Interaction,2017,33(6):510-522.
[4]Ficocelli M,Terao J,Nejat G.Promoting Interactions between Humans and Robots Using Robotic Emotional Behavior[J].IEEE Trans Cybern,2016,46(12):2911-2923.
[5]Nayak V,Turk M.Emotional Expression in Virtual Agents through Body Language[M].Springer Berlin Heidelberg,2005.
[6]Bonarini A.Can My Robotic Home Cleaner Be Happy Issues about Emotional Expression in Non-bio-inspired Robots[J].Adaptive Behavior,2016,24(5):335-349.
[7]Sial S B,Sial M B,Ayaz Y,et al.Interaction of Robot with Humans by Communicating Simulated Emotional States through Expressive Movements[J].Intelligent Service Robotics,2016,9(3):231-255.
[8]Bretan M,Hoffman G,Weinberg G.Emotionally Expressive Dynamic Physical Behaviors in Robots[J].International Journal of Human-Computer Studies,2015,78(C):1-16.
[9]Novikova J,Watts L.A Design Model of Emotional Body Expressions in Non-humanoid Robots[C]//Proceedings of the Second International Conference on Human-agent Interaction,2014:353-360.
[10]Jung M F.Affective Grounding in Human-Robot Interaction[C]//Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction,2017:263-273.
[11]Bethel C L,Murphy R R.Review of Human Studies Methods in HRI and Recommendations[J].International Journal of Social Robotics,2010,2(4):347-359.
[12]Partala T,Surakka V.Pupil Size Variation as An Indication of Affective Processing[J].International Journal of Human-Computer Studies,2003,59(1-2):185-198.
[13]李强,侯艳红,苗丹民,等.不同情绪模式图片眼动特征的研究[J].中国行为医学科学,2006,15(12):1121-1122.
[14]Janisse M P.Pupil Size,Affect and Exposure Frequency[J].Social Behavior and Personality,1974,2(2):125-146.
[15]Bekkedal M Y,Rossi J,Panksepp J.Human Brain EEG Indices of Emotions:Delineating Responses to Affective Vocalizations by Measuring Frontal Theta Event-related Synchronization[J].Neurosci Biobehav Rev,2011,35(9):1959-1970.
[16]Messerotti Benvenuti S,Mennella R,Buodo G,et al.Frontal Theta Activity as An EEG Correlate of MoodRelated Emotional Processing in Dysphoria[J].Journal of Psychopathology and Behavioral Assessment,2016,39(2):241-252.
[17]Coan J A,Allen J J.Frontal EEG Asymmetry as A Moderator and Mediator of Emotion[J].Biological Psychology,2004,67(1-2):7-49.
[18]Tsang C D,Trainor L J,Santesso D L,et al.Frontal EEG Responses as A Function of Affective Musical Features[J].Biological Foundations of Music,2001,930(1):439-442.
[19]Weinreich A,Stephani T,Schubert T.Emotion Effects within Frontal Alpha Oscillation in A Picture Oddball Paradigm[J].International Journal of Psychophysiology,2016,110:200-206.
[20]Mori M,Mac Dorman K F,Kageki N.The Uncanny Valley[J].Energy,1970,7(4):33-35.
[21]Lang P J.Behavioral Treatment and Bio-behavioral Assessment:Computer Applications[J].Technology in Mental Health Care Delivery Systems[M].Norwood Ablex,1980.
[22]卢英俊,戴丽丽,吴海珍,等.不同类型音乐对悲伤情绪舒缓作用的EEG研究[J].心理学探新,2012,32(04):369-375.
[23]Kim Y E,Schmidt E M,Migneco R,et al.Music Emotion Recognition:A State of the Art Review[C]//Proc ISMIR,2010:255-266.
[24]Roxas J C,Richards D,Bilgin A,et al.Exploring the Influence of A Human-like Dancing Virtual Character on the Evocation of Human Emotion[J].Behaviour&Information Technology,2018,37(1)1-15.
[25]Bradley M M,Miccoli L,Escrig M A,et al.The Pupil as A Measure of Emotional Arousal and Autonomic Activation[J].Psychophysiology,2008,45(4):602-607.
[26]Hess E H.Attitude and Pupil Size[J].Sci Am,1965,212(4):46-54.
[27]Laufs H,Kleinschmidt A,Beyerle A,et al.EEG-correlated FMRI of Human Alpha Activity[J].Neuroimage,2003,19(4):1463-1476.
[28]Goldman R I,Stern J M,Engel Jr J,et al.Simultaneous EEG and f MRI of the Alpha Rhythm[J].Neuroreport,2002,13(18):2487.
[2]Breazeal C.Emotion and Sociable Humanoid Robots[J].International Journal of Human-Computer Studies,2003,59(1-2):119-155.
[3]Khosla R,Nguyen K,Chu M T.Human Robot Engagement and Acceptability in Residential Aged Care[J].International Journal of Human-Computer Interaction,2017,33(6):510-522.
[4]Ficocelli M,Terao J,Nejat G.Promoting Interactions between Humans and Robots Using Robotic Emotional Behavior[J].IEEE Trans Cybern,2016,46(12):2911-2923.
[5]Nayak V,Turk M.Emotional Expression in Virtual Agents through Body Language[M].Springer Berlin Heidelberg,2005.
[6]Bonarini A.Can My Robotic Home Cleaner Be Happy Issues about Emotional Expression in Non-bio-inspired Robots[J].Adaptive Behavior,2016,24(5):335-349.
[7]Sial S B,Sial M B,Ayaz Y,et al.Interaction of Robot with Humans by Communicating Simulated Emotional States through Expressive Movements[J].Intelligent Service Robotics,2016,9(3):231-255.
[8]Bretan M,Hoffman G,Weinberg G.Emotionally Expressive Dynamic Physical Behaviors in Robots[J].International Journal of Human-Computer Studies,2015,78(C):1-16.
[9]Novikova J,Watts L.A Design Model of Emotional Body Expressions in Non-humanoid Robots[C]//Proceedings of the Second International Conference on Human-agent Interaction,2014:353-360.
[10]Jung M F.Affective Grounding in Human-Robot Interaction[C]//Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction,2017:263-273.
[11]Bethel C L,Murphy R R.Review of Human Studies Methods in HRI and Recommendations[J].International Journal of Social Robotics,2010,2(4):347-359.
[12]Partala T,Surakka V.Pupil Size Variation as An Indication of Affective Processing[J].International Journal of Human-Computer Studies,2003,59(1-2):185-198.
[13]李强,侯艳红,苗丹民,等.不同情绪模式图片眼动特征的研究[J].中国行为医学科学,2006,15(12):1121-1122.
[14]Janisse M P.Pupil Size,Affect and Exposure Frequency[J].Social Behavior and Personality,1974,2(2):125-146.
[15]Bekkedal M Y,Rossi J,Panksepp J.Human Brain EEG Indices of Emotions:Delineating Responses to Affective Vocalizations by Measuring Frontal Theta Event-related Synchronization[J].Neurosci Biobehav Rev,2011,35(9):1959-1970.
[16]Messerotti Benvenuti S,Mennella R,Buodo G,et al.Frontal Theta Activity as An EEG Correlate of MoodRelated Emotional Processing in Dysphoria[J].Journal of Psychopathology and Behavioral Assessment,2016,39(2):241-252.
[17]Coan J A,Allen J J.Frontal EEG Asymmetry as A Moderator and Mediator of Emotion[J].Biological Psychology,2004,67(1-2):7-49.
[18]Tsang C D,Trainor L J,Santesso D L,et al.Frontal EEG Responses as A Function of Affective Musical Features[J].Biological Foundations of Music,2001,930(1):439-442.
[19]Weinreich A,Stephani T,Schubert T.Emotion Effects within Frontal Alpha Oscillation in A Picture Oddball Paradigm[J].International Journal of Psychophysiology,2016,110:200-206.
[20]Mori M,Mac Dorman K F,Kageki N.The Uncanny Valley[J].Energy,1970,7(4):33-35.
[21]Lang P J.Behavioral Treatment and Bio-behavioral Assessment:Computer Applications[J].Technology in Mental Health Care Delivery Systems[M].Norwood Ablex,1980.
[22]卢英俊,戴丽丽,吴海珍,等.不同类型音乐对悲伤情绪舒缓作用的EEG研究[J].心理学探新,2012,32(04):369-375.
[23]Kim Y E,Schmidt E M,Migneco R,et al.Music Emotion Recognition:A State of the Art Review[C]//Proc ISMIR,2010:255-266.
[24]Roxas J C,Richards D,Bilgin A,et al.Exploring the Influence of A Human-like Dancing Virtual Character on the Evocation of Human Emotion[J].Behaviour&Information Technology,2018,37(1)1-15.
[25]Bradley M M,Miccoli L,Escrig M A,et al.The Pupil as A Measure of Emotional Arousal and Autonomic Activation[J].Psychophysiology,2008,45(4):602-607.
[26]Hess E H.Attitude and Pupil Size[J].Sci Am,1965,212(4):46-54.
[27]Laufs H,Kleinschmidt A,Beyerle A,et al.EEG-correlated FMRI of Human Alpha Activity[J].Neuroimage,2003,19(4):1463-1476.
[28]Goldman R I,Stern J M,Engel Jr J,et al.Simultaneous EEG and f MRI of the Alpha Rhythm[J].Neuroreport,2002,13(18):2487.