视频中非特定人的面部表情分析算法
|
摘要
针对视频人脸表情识别存在对非特定人的面部表情识别率低(约75%),有效的特征信息提取困难等问题,提出了基于分块的三维正交平面的完全局部二值模式(CLBP-TOP)的特征提取方法,实现了对任意长度视频的面部表情识别和分析。对表情视频序列进行预处理;对人脸主要区域的每个分块提取CLBP-TOP特征,得到最后的联合统计直方图特征;利用动态时间规划度量距离的最近邻规则进行表情识别;提出的表情持续时间和表情强度指数实现了对表情分析的详细描述。在CK+数据库上测试,提出的方法与LBP-TOP方法相比平均识别率提高了12. 04%,且能够对任意长度的表情视频片段进行表情识别和相关系数分析。
In view of the fact that there are problems that facial expression recognition rate of non-specific person is low( about 75 %),and extraction of effective feature information is difficult,a feature extraction method for completed local binary pattern of 3 D orthogonal planes( CLBP-TOP) based on block is proposed and facial expression recognition of arbitrary length of the video is achieved. Expression video sequence is preprocessed,and CLBP-TOP feature is extracted from each block of main face region,and the final joint statistical histogram feature is obtained. The nearest neighbor rules of the dynamic time warping is used for expression recognition. The time of duration and expression intensity scores achieve the detailed description of the expression analysis. Test on CK +database shows that the average recognition rate of this new method increases 12. 04 % compare to CLBP-TOP,and facial expression recognition and correlation coefficient analysis of any length of facial expression video clips can be performed.
In view of the fact that there are problems that facial expression recognition rate of non-specific person is low( about 75 %),and extraction of effective feature information is difficult,a feature extraction method for completed local binary pattern of 3 D orthogonal planes( CLBP-TOP) based on block is proposed and facial expression recognition of arbitrary length of the video is achieved. Expression video sequence is preprocessed,and CLBP-TOP feature is extracted from each block of main face region,and the final joint statistical histogram feature is obtained. The nearest neighbor rules of the dynamic time warping is used for expression recognition. The time of duration and expression intensity scores achieve the detailed description of the expression analysis. Test on CK +database shows that the average recognition rate of this new method increases 12. 04 % compare to CLBP-TOP,and facial expression recognition and correlation coefficient analysis of any length of facial expression video clips can be performed.
引文
[1]Valstar M F,Mehu M,Jiang B,et al.Meta-analysis of the first facial expression recognition challenge[J].IEEE Transactions on Systems Man&Cybernetics Part B:Cybernetics A Publication of the IEEE Systems Man&Cybernetics Society,2012,42(4):966-979.
[2]刘清泉,张亚飞,李华锋,等.基于协作低秩分层稀疏和LC-KSVD的人脸表情识别[J].传感器与微系统,2017,36(11):56-59.
[3]梅英,谭冠政,刘振焘.基于视频图像的面部表情识别研究综述[J].湖南文理学院学报:自然科学版,2016,28(3):19-25.
[4]Ekman P,Friesen W V.Constants across cultures in the face and emotion[J].Journal of Personality and Social Psychology,1971,17(2):124.
[5]Cootes T,Edwards G J,Taylor C J.Active appearance models[C]∥The 5th European Conference on Computer Vision,Freiburg,Germany,1998:484-498.
[6]Jin Q,Zhao J,Zhang Y.Facial feature extraction with a depth AAM algorithm[C]∥International Conference on Fuzzy Systems and Knowledge Discovery,IEEE,2012:1792-1796.
[7]Huang F C,Huang S Y,KER J W,et al.High-performance SIFThardware accelerator for real-time image feature extraction[J].IEEE Transactions on Circuits&Systems for Video Technology,2012,22(3):340-351.
[8]杨瑞,张云伟,苟爽,等.Gabor特征与深度信念网络结合的人脸识别方法[J].传感器与微系统,2017,36(5):68-70.
[9]Khorsheed J A,Yurtkan K.Analysis of local binary patterns for face recognition under varying facial expressions[C]∥Signal Processing and Communication Application Conference,IEEE,2016:2085-2088.
[10]Guo Z,Zhang L,Zhang D.A completed modeling of local binary pattern operator for texture classification.[J].IEEE Transactions on Image Processing,2010,19(6):1657-1663.
[11]胡敏,许艳侠,王晓华,等.自适应加权完全局部二值模式的表情识别[J].中国图象图形学报,2013,18(10):1279-1284.
[12]Banu S M,Danciu G M,Boboc R G,et al.A novel approach for face expressions recognition[C]∥IEEE Jubilee International Symposium on Intelligent Systems and Informatics,IEEE,2012:537-541.
[13]易积政,毛峡,Ishizuka Mitsuru,等.基于特征点矢量与纹理形变能量参数融合的人脸表情识别[J].电子与信息学报,2013,35(10):2403-2410.
[14]Hamester D,Barros P,Wermter S.Face expression recognition with a 2-channel convolutional neural network[C]∥International Joint Conference on Neural Networks,IEEE,2015:1-8.
[15]Zhao G Y,Pietikainen M.Dynamic texture recognition using local binary patterns with an application to facial expressions[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2007,29(6):915-928.
[2]刘清泉,张亚飞,李华锋,等.基于协作低秩分层稀疏和LC-KSVD的人脸表情识别[J].传感器与微系统,2017,36(11):56-59.
[3]梅英,谭冠政,刘振焘.基于视频图像的面部表情识别研究综述[J].湖南文理学院学报:自然科学版,2016,28(3):19-25.
[4]Ekman P,Friesen W V.Constants across cultures in the face and emotion[J].Journal of Personality and Social Psychology,1971,17(2):124.
[5]Cootes T,Edwards G J,Taylor C J.Active appearance models[C]∥The 5th European Conference on Computer Vision,Freiburg,Germany,1998:484-498.
[6]Jin Q,Zhao J,Zhang Y.Facial feature extraction with a depth AAM algorithm[C]∥International Conference on Fuzzy Systems and Knowledge Discovery,IEEE,2012:1792-1796.
[7]Huang F C,Huang S Y,KER J W,et al.High-performance SIFThardware accelerator for real-time image feature extraction[J].IEEE Transactions on Circuits&Systems for Video Technology,2012,22(3):340-351.
[8]杨瑞,张云伟,苟爽,等.Gabor特征与深度信念网络结合的人脸识别方法[J].传感器与微系统,2017,36(5):68-70.
[9]Khorsheed J A,Yurtkan K.Analysis of local binary patterns for face recognition under varying facial expressions[C]∥Signal Processing and Communication Application Conference,IEEE,2016:2085-2088.
[10]Guo Z,Zhang L,Zhang D.A completed modeling of local binary pattern operator for texture classification.[J].IEEE Transactions on Image Processing,2010,19(6):1657-1663.
[11]胡敏,许艳侠,王晓华,等.自适应加权完全局部二值模式的表情识别[J].中国图象图形学报,2013,18(10):1279-1284.
[12]Banu S M,Danciu G M,Boboc R G,et al.A novel approach for face expressions recognition[C]∥IEEE Jubilee International Symposium on Intelligent Systems and Informatics,IEEE,2012:537-541.
[13]易积政,毛峡,Ishizuka Mitsuru,等.基于特征点矢量与纹理形变能量参数融合的人脸表情识别[J].电子与信息学报,2013,35(10):2403-2410.
[14]Hamester D,Barros P,Wermter S.Face expression recognition with a 2-channel convolutional neural network[C]∥International Joint Conference on Neural Networks,IEEE,2015:1-8.
[15]Zhao G Y,Pietikainen M.Dynamic texture recognition using local binary patterns with an application to facial expressions[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2007,29(6):915-928.