基于视觉机制与协同显著性的自发式表情识别
|
摘要
自发性面部表情视频中存在过多的冗余图像数据而在性能上遇到瓶颈,为此提出一种基于人眼视觉注意机制和协同显著性人脸特征提取算法。基于人类的视觉机制,动态提取关键帧解决面部表情的时间动态问题。通过乘法融合算法和图像协同获得关键序列帧显著区域。由于这些区域形变较小且与面部表情低相关,剔除这些显著区域,减少面部特征数据量。提取剩余面部的LBP特征,并使用SVM分类器进行分类。在Cohn-Kanade+人脸表情数据库和MMI数据库中的实验结果表明,该方法能有效提高自发表达序列的识别准确率。
Most relative algorithms encounter bottle-necks in performance due to big redundant images data in the video.A co-salient facial feature extraction algorithm was proposed,based on human visual attention mechanism and group data co-processing technology.Based on human visual mechanism,key frames of expression were dynamically derived from sampling videos to solve the temporal dynamics of facial expressions.Using key sequence frames,salient regions were obtained using multiplicative fusion algorithm in multi-images co-operative manner.These salient regions were got rid of due to their little deformation and low-correlation to facial expressions,and the amount of facial features data was reduced.LBP features of the remainder of facial features were extracted and SVM classifier was used to classify them,respectively.In Cohn-Kanade+facial expression database and MMI database,experimental results show that the proposed method can effectively improve the recognizing accuracy of spontaneous expression sequence.
Most relative algorithms encounter bottle-necks in performance due to big redundant images data in the video.A co-salient facial feature extraction algorithm was proposed,based on human visual attention mechanism and group data co-processing technology.Based on human visual mechanism,key frames of expression were dynamically derived from sampling videos to solve the temporal dynamics of facial expressions.Using key sequence frames,salient regions were obtained using multiplicative fusion algorithm in multi-images co-operative manner.These salient regions were got rid of due to their little deformation and low-correlation to facial expressions,and the amount of facial features data was reduced.LBP features of the remainder of facial features were extracted and SVM classifier was used to classify them,respectively.In Cohn-Kanade+facial expression database and MMI database,experimental results show that the proposed method can effectively improve the recognizing accuracy of spontaneous expression sequence.
引文
[1]HE Menghua.Spontaneous and human face expression analysis[D].Hefei:University of Science and Technology of China,2014:15-16(in Chinese).[何孟华.自发和人为表情分析识别研究[D].合肥:中国科学技术大学,2014:15-16.]
[2]YE Jianfeng,WANG Huaming.AdaBoost detects automatic face recognition method combined with SOM[J].Journal of Harbin Engineering University,2018,39(1):129-134(in Chinese).[叶剑锋,王化明.AdaBoost检测结合SOM的自动人脸识别方法[J].哈尔滨工程大学学报,2018,39(1):129-134.]
[3]XU Feng,ZHANG Junping.A survey of face micro-expression recognition[J].Journal of Automation,2017,43(3):333-348(in Chinese).[徐峰,张军平.人脸微表情识别综述[J].自动化学报,2017,43(3):333-348.]
[4]ZHAO Wen,WU Xiaojun.Robust face recognition algorithm based on differential low-rank representation and dictionary learning[J].Application Research of Computers,2017,34(10):3157-3161(in Chinese).[赵雯,吴小俊.基于鉴别性低秩表示及字典学习的鲁棒人脸识别算法[J].计算机应用研究,2017,34(10):3157-3161.]
[5]HUANG Faliang,YU Ge,ZHANG Jilian.Weibo theme emotional mining based on social relationships[J].Journal of Software,2017,28(3):694-707(in Chinese).[黄发良,于戈,张继连.基于社交关系的微博主题情感挖掘[J].软件学报,2017,28(3):694-707.]
[6]WU Dongyin,GUI Lin,CHEN Zhao.Sentiment analysis method based on depth representation learning and Gaussian process transfer learning[J].Chinese Journal of Information,2017,31(1):169-176(in Chinese).[吴冬茵,桂林,陈钊.基于深度表示学习和高斯过程迁移学习的情感分析方法[J].中文信息学报,2017,31(1):169-176.]
[7]TANG Xiaobo,LIU Guangchao.Research summary of finegrained sentiment analysis[J].Library and Information Work,2017,61(5):132-140(in Chinese).[唐晓波,刘广超.细粒度情感分析研究综述[J].图书情报工作,2017,61(5):132-140.]
[8]Guo W,You S,Gao J,et al.Deep relative metric learning for visual tracking[J].SCIENTIA SINICA Informationis,2018,48(1):60-78.
[9]Jiang B,Valstar M,Martinez B,et al.A dynamic appearance descriptor approach to facial actions temporal modeling[J].IEEE Transactions on Cybernetics,2014,44(2):161-174.
[10]Yan B,Yu S,Pei T,et al.Vision interaction method based on visual attention mechanism[C]//12th IEEE Conference on Industrial Electronics and Applications.IEEE,2017:930-935.
[11]MAO Ling,ZHANG Guomin.High-resolution remote sensing image port detection composite clue visual attention mode[J].Journal of Remote Sensing,2017,21(2):300-309(in Chinese).[毛玲,张国敏.高分辨率遥感图像港口检测的复合线索视觉注意模型[J].遥感学报,2017,21(2):300-309.]
[12]Moore T,Zirnsak M.Neural mechanisms of selective visual attention[J].Annual Review of Psychology,2017,68:47-72.
[13]Huazhu F,Xiao C,Zhuowen T.Cluster-based cosaliency detection[J].IEEE Transactions on Image Processing,2013,22(10):3766-3778.
[14]LI Shuangshuang,ZHAO Gaopeng,WANG Jianyu.Interference-aware target tracking based on feature fusion and scale adaptive[J].Journal of Optics,2017,37(5):208-217(in Chinese).[李双双,赵高鹏,王建宇.基于特征融合和尺度自适应的干扰感知目标跟踪[J].光学学报,2017,37(5):208-217.]
[15]HU Yanxiang,WAN Li.Heterogeneous image fusion based on visual attention mechanism[J].Computer Engineerng,2015,41(3):247-252(in Chinese).[胡燕翔,万莉.基于视觉注意机制的异源图像融合[J].计算机工程,2015,41(3):247-252.]
[16]SHAO Hong,WANG Yang.Dynamic sequence expression recognition based on AAM and optical flow method[J].Computer Engineering and Design,2017,38(6):1642-1656(in Chinese).[邵虹,王洋.基于AAM和光流法的动态序列表情识别[J].计算机工程与设计,2017,38(6):1642-1656.]
[2]YE Jianfeng,WANG Huaming.AdaBoost detects automatic face recognition method combined with SOM[J].Journal of Harbin Engineering University,2018,39(1):129-134(in Chinese).[叶剑锋,王化明.AdaBoost检测结合SOM的自动人脸识别方法[J].哈尔滨工程大学学报,2018,39(1):129-134.]
[3]XU Feng,ZHANG Junping.A survey of face micro-expression recognition[J].Journal of Automation,2017,43(3):333-348(in Chinese).[徐峰,张军平.人脸微表情识别综述[J].自动化学报,2017,43(3):333-348.]
[4]ZHAO Wen,WU Xiaojun.Robust face recognition algorithm based on differential low-rank representation and dictionary learning[J].Application Research of Computers,2017,34(10):3157-3161(in Chinese).[赵雯,吴小俊.基于鉴别性低秩表示及字典学习的鲁棒人脸识别算法[J].计算机应用研究,2017,34(10):3157-3161.]
[5]HUANG Faliang,YU Ge,ZHANG Jilian.Weibo theme emotional mining based on social relationships[J].Journal of Software,2017,28(3):694-707(in Chinese).[黄发良,于戈,张继连.基于社交关系的微博主题情感挖掘[J].软件学报,2017,28(3):694-707.]
[6]WU Dongyin,GUI Lin,CHEN Zhao.Sentiment analysis method based on depth representation learning and Gaussian process transfer learning[J].Chinese Journal of Information,2017,31(1):169-176(in Chinese).[吴冬茵,桂林,陈钊.基于深度表示学习和高斯过程迁移学习的情感分析方法[J].中文信息学报,2017,31(1):169-176.]
[7]TANG Xiaobo,LIU Guangchao.Research summary of finegrained sentiment analysis[J].Library and Information Work,2017,61(5):132-140(in Chinese).[唐晓波,刘广超.细粒度情感分析研究综述[J].图书情报工作,2017,61(5):132-140.]
[8]Guo W,You S,Gao J,et al.Deep relative metric learning for visual tracking[J].SCIENTIA SINICA Informationis,2018,48(1):60-78.
[9]Jiang B,Valstar M,Martinez B,et al.A dynamic appearance descriptor approach to facial actions temporal modeling[J].IEEE Transactions on Cybernetics,2014,44(2):161-174.
[10]Yan B,Yu S,Pei T,et al.Vision interaction method based on visual attention mechanism[C]//12th IEEE Conference on Industrial Electronics and Applications.IEEE,2017:930-935.
[11]MAO Ling,ZHANG Guomin.High-resolution remote sensing image port detection composite clue visual attention mode[J].Journal of Remote Sensing,2017,21(2):300-309(in Chinese).[毛玲,张国敏.高分辨率遥感图像港口检测的复合线索视觉注意模型[J].遥感学报,2017,21(2):300-309.]
[12]Moore T,Zirnsak M.Neural mechanisms of selective visual attention[J].Annual Review of Psychology,2017,68:47-72.
[13]Huazhu F,Xiao C,Zhuowen T.Cluster-based cosaliency detection[J].IEEE Transactions on Image Processing,2013,22(10):3766-3778.
[14]LI Shuangshuang,ZHAO Gaopeng,WANG Jianyu.Interference-aware target tracking based on feature fusion and scale adaptive[J].Journal of Optics,2017,37(5):208-217(in Chinese).[李双双,赵高鹏,王建宇.基于特征融合和尺度自适应的干扰感知目标跟踪[J].光学学报,2017,37(5):208-217.]
[15]HU Yanxiang,WAN Li.Heterogeneous image fusion based on visual attention mechanism[J].Computer Engineerng,2015,41(3):247-252(in Chinese).[胡燕翔,万莉.基于视觉注意机制的异源图像融合[J].计算机工程,2015,41(3):247-252.]
[16]SHAO Hong,WANG Yang.Dynamic sequence expression recognition based on AAM and optical flow method[J].Computer Engineering and Design,2017,38(6):1642-1656(in Chinese).[邵虹,王洋.基于AAM和光流法的动态序列表情识别[J].计算机工程与设计,2017,38(6):1642-1656.]