结合面部纹理和光流特征的面瘫分级综合评估方法
|
摘要
针对面瘫分级评估问题,提出一种结合面部纹理和光流特征的面瘫分级综合评估方法。该方法针对面瘫诊断时患者需要完成不同的面部表情动作,关注并提取对应的面部区域特征,以及人脸两侧相关区域特征之间的差异以完成面瘫分级评估。首先,需要对面瘫静态图像和视频数据进行预处理;然后,利用人脸关键点检测方法对人脸进行区域划分;接着,针对面瘫图像和视频数据,分别依据人脸左右两侧对应区域的纹理特征差异和光流特征差异以完成面瘫分级评估;最后,利用基于图像和视频数据的评估结果进行面瘫分级的综合评估。实验表明,所提方法的面瘫分级评估平均准确率相对于传统方法提高了18%以上,具有明显优势。
For the evaluation of facial nerve paralysis,a comprehensive evaluation method for facial paralysis,which utilizes the facial texture and optical flow features is proposed. Considering that the patients need to do several facial expressions for the diagnosis of facial paralysis,several types of facial features in the corresponding facial areas are extracted,and then the differences between the features of the related regions on both sides of the face are used to complete the evaluation of facial paralysis. Firstly,the static image and videos for each patient's diagnosis is preprocessed. Then the key point detection is performed on the face to divide the whole face into several facial regions. Further,according to the differences of texture features and optical flow features of the corresponding regions on the left and right sides of the face,the evaluation of facial paralysis can be completed for the image and videos. Finally,the comprehensive evaluation of facial paralysis is carried out by combining the evaluation results from the image and videos for a patient's diagnosis. The experimental results show that the average accuracy of the evaluation of facial paralysis is improved by more than 18% compared with traditional methods,which means the proposed method has obvious advantages.
For the evaluation of facial nerve paralysis,a comprehensive evaluation method for facial paralysis,which utilizes the facial texture and optical flow features is proposed. Considering that the patients need to do several facial expressions for the diagnosis of facial paralysis,several types of facial features in the corresponding facial areas are extracted,and then the differences between the features of the related regions on both sides of the face are used to complete the evaluation of facial paralysis. Firstly,the static image and videos for each patient's diagnosis is preprocessed. Then the key point detection is performed on the face to divide the whole face into several facial regions. Further,according to the differences of texture features and optical flow features of the corresponding regions on the left and right sides of the face,the evaluation of facial paralysis can be completed for the image and videos. Finally,the comprehensive evaluation of facial paralysis is carried out by combining the evaluation results from the image and videos for a patient's diagnosis. The experimental results show that the average accuracy of the evaluation of facial paralysis is improved by more than 18% compared with traditional methods,which means the proposed method has obvious advantages.
引文
[1]王倩倩.基于AAM的面神经运动功能评价方法研究[D].青岛:中国海洋大学,2012.
[2]WACHTMAN G S,COHN J F,VANSWEARINGEN JM,et al.Automated tracking of facial features in patients with facial neuromuscular dysfunction[J].Plastic&Reconstructive Surgery,2001,107(5):1124-1133.
[3]MODERSOHN L,DENZLER J.Facial paresis index prediction by exploiting active appearance models for compact discriminative features[C]∥International Conference on Computer Vision Theory and Applications.INSTICC,2016:271-278.
[4]HE S,SORAGHAN J J,O'REILLY B F,et al.Quantitative analysis of facial paralysis using local binary patterns in biomedical videos[J].IEEE Transactions on Bio-Medical Engineering,2009,6(7):1864-1870.
[5]SHU H,SORAGHAN J J,BRIAN F O.Biomedical image sequence analysis with application to automatic quantitative assessment of facial paralysis[J].Eurasip Journal on Image&Video Processing,2007,3(7):081282.
[6]BARBOSA J,LEE K,LEE S,et al.Efficient quantitative assessment of facial paralysis using iris segmentation and active contour-based key points detection with hybrid classifier[J].BMC Medical Imaging,2016,16(1):23.
[7]WANG T,ZHANG S,DONG J,et al.Automatic evaluation of the degree of facial nerve paralysis[J].Multimedia Tools and Applications,2016,75(19):11893-11908.
[8]WANG S,LI H,QI F,et al.Objective facial paralysis grading based on Pface and eigenflow[J].Medical&Biological Engineering&Computing,2004,42(5):598-603.
[9]SMEETS D,CLAES P,HERMANS J,et al.A comparative study of 3-D face recognition under expression variations[J].IEEE Transactions on Systems Man&Cybernetics Part C,2012,42(5):710-727.
[10]DONATO G,BARTLETT M S,HAGER J C,et al.Classifying facial actions[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1999,21(10):974-989.
[11]CHEN J,CHEN Z,CHI Z,et al.Facial expression recognition in video with multiple feature fusion[J].IEEE Transactions on Affective Computing,2018(99):38-50.
[12]DELPIANO J,PIZARRO L,VERSCHAE R,et al.Multi-objective optimization for parameter selection and characterization of optical flow methods[J].Applied Soft Computing,2016,46(C):1067-1078.
[13]FISCHER T,KRAUSS C.Deep learning with long short-term memory networks for financial market predictions[J].European Journal of Operational Research,2018,270(2):654-669.
[14]ZHAO R,GAN Q,WANG S,et al.Facial expression intensity estimation using ordinal information[C]∥2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE Computer Society,2016:3466-3474.
[15]FENG X,PIETIKINEN M,HADID A.Facial expression recognition based on local binary patterns[J].Pattern Recognition and Image Analysis,2007,17(4):592-598.
[16]ZHANG Z,LYONS M,SCHUSTER M,et al.Comparison between geometry-based and Gabor waveletsbased facial expression recognition using multi-layer perceptron[C]∥Third IEEE International Conference on Automatic Face and Gesture Recognition.IEEEComputer Society,1998:454-459.
[17]ENKELMANN W.Investigations of multigrid algorithms for the estimation of optical flow fields in image sequences[J].Computer Vision,Graphics and Image Processing,1988,42(3):410-411.
[18]YUAN G W,CHEN Z Q,GONG J,et al.A Moving object detection algorithm based on a combination of optical flow and three-frame difference[J].Journal of Chinese Computer Systems,2013,34(3):668-671.
[2]WACHTMAN G S,COHN J F,VANSWEARINGEN JM,et al.Automated tracking of facial features in patients with facial neuromuscular dysfunction[J].Plastic&Reconstructive Surgery,2001,107(5):1124-1133.
[3]MODERSOHN L,DENZLER J.Facial paresis index prediction by exploiting active appearance models for compact discriminative features[C]∥International Conference on Computer Vision Theory and Applications.INSTICC,2016:271-278.
[4]HE S,SORAGHAN J J,O'REILLY B F,et al.Quantitative analysis of facial paralysis using local binary patterns in biomedical videos[J].IEEE Transactions on Bio-Medical Engineering,2009,6(7):1864-1870.
[5]SHU H,SORAGHAN J J,BRIAN F O.Biomedical image sequence analysis with application to automatic quantitative assessment of facial paralysis[J].Eurasip Journal on Image&Video Processing,2007,3(7):081282.
[6]BARBOSA J,LEE K,LEE S,et al.Efficient quantitative assessment of facial paralysis using iris segmentation and active contour-based key points detection with hybrid classifier[J].BMC Medical Imaging,2016,16(1):23.
[7]WANG T,ZHANG S,DONG J,et al.Automatic evaluation of the degree of facial nerve paralysis[J].Multimedia Tools and Applications,2016,75(19):11893-11908.
[8]WANG S,LI H,QI F,et al.Objective facial paralysis grading based on Pface and eigenflow[J].Medical&Biological Engineering&Computing,2004,42(5):598-603.
[9]SMEETS D,CLAES P,HERMANS J,et al.A comparative study of 3-D face recognition under expression variations[J].IEEE Transactions on Systems Man&Cybernetics Part C,2012,42(5):710-727.
[10]DONATO G,BARTLETT M S,HAGER J C,et al.Classifying facial actions[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1999,21(10):974-989.
[11]CHEN J,CHEN Z,CHI Z,et al.Facial expression recognition in video with multiple feature fusion[J].IEEE Transactions on Affective Computing,2018(99):38-50.
[12]DELPIANO J,PIZARRO L,VERSCHAE R,et al.Multi-objective optimization for parameter selection and characterization of optical flow methods[J].Applied Soft Computing,2016,46(C):1067-1078.
[13]FISCHER T,KRAUSS C.Deep learning with long short-term memory networks for financial market predictions[J].European Journal of Operational Research,2018,270(2):654-669.
[14]ZHAO R,GAN Q,WANG S,et al.Facial expression intensity estimation using ordinal information[C]∥2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE Computer Society,2016:3466-3474.
[15]FENG X,PIETIKINEN M,HADID A.Facial expression recognition based on local binary patterns[J].Pattern Recognition and Image Analysis,2007,17(4):592-598.
[16]ZHANG Z,LYONS M,SCHUSTER M,et al.Comparison between geometry-based and Gabor waveletsbased facial expression recognition using multi-layer perceptron[C]∥Third IEEE International Conference on Automatic Face and Gesture Recognition.IEEEComputer Society,1998:454-459.
[17]ENKELMANN W.Investigations of multigrid algorithms for the estimation of optical flow fields in image sequences[J].Computer Vision,Graphics and Image Processing,1988,42(3):410-411.
[18]YUAN G W,CHEN Z Q,GONG J,et al.A Moving object detection algorithm based on a combination of optical flow and three-frame difference[J].Journal of Chinese Computer Systems,2013,34(3):668-671.