基于混合特征和神经网络集成的人脸表情识别
详细信息    本馆镜像全文|  推荐本文 |  |   获取CNKI官网全文
摘要
本文以JAFFE(Japanese Female Facial Expression)表情图库及本课题组组建的自然表情图库为研究对象,以克服光照因素与个体特征对表情的干扰为目标,进行人脸表情识别的研究。
     本文基于几何结构特征与局部统计特征相结合的混合特征方法进行表情识别。首先,本文首先对AdaBoost机器学习算法进行研究,定位出人脸区域以及表情变化形变量较大的特征区域。然后,提取几何结构特征和局部统计特征,利用AAM(Active Appearance Model)的方法定位出脸部关键特征点,将表情变化引起的特征点形变量作为几何结构特征;对眼睛、嘴部特征区域提取统计特征,并采用Fisher准则对其进行特征选择,从而得出最有效表征表情的特征。最后,通过神经网络集成分类器进行表情分类实验。实验结果验证了本文基于混合特征方法识别表情的有效性。
     本论文的研究得到了吉林省科技发展计划重要项目(20071152)的资助。
Facial expression plays an important role in daily life, it is a main way of human nonverbal communication, and it is an important supplement of exchanged language. Facial expression recognition is the basis of emotional understanding, and premise of computer understanding or expressing emotion of human. Recently, with the development of affective computing, facial expression recognition is become a very active research in scientific community.
     Facial expression recognition is a cross-subject in the fields of pattern recognition, machine vision, physiology and psychology. Because of the complexity and specificity of facial expression, which make facial expression recognition become one of the most challenging problems and have a broad application prospect.
     Generally, facial expression recognition system includes the following three parts: expression image preprocessing, expression feature extraction and classification. As feature extraction and expression classification are the key steps of facial expression, this paper has a deep research on these two aspects. The main work is as follows:
     (1) Expression image preprocessing.
     First, we need to locate human face and feature areas. Our study is based on JAFFE database. In order to locate human face, eyes and mouth accurately, AdaBoost is adopted in this paper. Experiment shows it not only can sure accurate locating, but can satisfy real-time system’s requirement on time. On the basis of locating face expression images, gray of expression images is equalized. After equalization the details of the image get clearer, and the distribution of gray levels of histogram gets evener. It also overcome great difference between gray levels of the same expression due to illumination, and make sure that learning and testing images are in the same condition.
     (2) Expression feature extraction.
     Feature extraction is the most important steps which result in the classification rate of the facial expression. When facial expression appears, it must cause facial deformation which includes the information for the expression classification. Therefore, this paper will denote these deformation by extracting the geometry structural features which reflect the change of facial shape and the local statistic features which reflect the change of facial texture, and eliminate redundant features as possible, in other words, we just use the necessary facial features for expression recognition. In order to extract geometry structural features, the method of AAM (Active Appearance Model) is used to locate feature points in the facial images, and extract 12 features which reflect facial deformation. To avoid the loss of the information, we generate local statistic feature using co-occurrence matrix as the supplement, more vividly describe the relationship amount the pixel and the statistic feature, in this paper we select the most effective statistic features to denote expression based on Fisher criterion. Experiment shows that the hybrid features can conquer the interference factor of the individual feature and the sunshine.
     (3) Facial expression classification.
     Because of the diversity and complexity of facial expression, it’s difficult to classify the facial expression by linear classification. In this paper, we bring neural network ensembles classification into expression recognition. Neural network ensembles classifier which we build includes 3 sub-networks, each sub-network bases on RBF network. We choose 137 images from JAFFE database as the training samples, and rest 60 images to test. For the following seven expressions:”Angry”、”Disgust”、”Fear”、”Happy”、”Neutral”、”Sad”and”Surprise”, experiments show that the average recognition rate can reach to 85.53% and 88.16% using single RBF classification and Neural network ensembles classification, the classification effect of ensembles classifier is better than single RBF classifier, and it improves the generalization ability of classifier. In order to strengthen the reliability of the algorithm in this paper, we choose 48 images from JLUFE database, and we also achieve very well recognition effect.
引文
[1]史于心.基于混合特征的面部表情识别算法研究[D].吉林大学硕士学位论文.2007, 04.
    [2] Sung K.-K, Poggio T. Example-based learning for view-based human face detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1998, 20(1): 39-51.
    [3] Turk M, Pentland A. Eigenfaces for recognition[J]. Journal of cognitive neuro science. 1991, 3(1): 71-86.
    [4] Pentland A, Moghaddam B, Starner T. View-based and modular eigenspaces for face recognition[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA. 1994: 21-23.
    [5] Moghaddam B, Pentland A. Face recognition using view-based and modular eigenspaces[C]. Proceedings of SPIE 2257.1994: 12-21.
    [6] Henry A. Rowley, Baluja S, Kanade T. Neural Network-Based Face Detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1998, 20(1): 23-38.
    [7] Henry A. Rowley , Baluja S, Kanade T. Rotation invariant neural network-based face detection[C]. Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition. 1998: 38-44.
    [8] Guangzheng Yang, Thomas S. Huang. Human face detection in a complex background[J]. Pattern Recognition.1994, 27(1): 53-63.
    [9] Kin Choong Yow, Roberto Cipolla.Feature-based human face detection[J]. Image and Vision Computing. 1997, 15(9): 713-735.
    [10] Michael C. Burl, Markus Weber, Pietro Perona. A probabilistic approach to object recognition using local photometry and global geometry[C]. Proceedings of the European Conference on Computer Vision. 1998: 628-641.
    [11] Yullie A, Hallinan P, Cohen D. Feature exaction from faces using deformable templates[J]. International Journal of Computing Vision. 1992, 8(2): 99-111.
    [12] Fang Liu, Zhi-liang Wang, Li Wang, et al. Facial Expression Recognition Using HLAC Features and WPCA[C]. LNCS 3784. 2005: 88-94.
    [13]张林,胡波.基于主元分析和Fuzzy ART模型的人脸识别算法[J].电路与系统学报.1999, 4(3): 9-17.
    [14] Gottumukkal R, Asari V K. An Improved Face RecognitionTechnique Based on Modular PCA Approach[J]. Pattern Recognition Letters, 2004, 25(4): 429-436.
    [15] Qingzhang Chen, Weiyi Zhang, Xiaoying Chen, et al. A Facial Expression Classifica-tion Algorithm Based on Principle Component Analysis[C]. Lecture Notes in Computer Science. 2006, Vol.3972: 55-62.
    [16] Calder Andrew J, Burton A Mike, Miller Paul, et al. A Principal Component Analysis of Facial Expressions [J]. Vision research. 2001, 41: 1179-1208.
    [17]叶敬福,詹永照.基于Gabor小波变换的人脸表情特征提取[J].计算机工程.2005, 31(15): 172-174.
    [18]周艳平,夏利民,宋星光.基于Gabor滤波的多分类器集成人脸表情识别[J].长沙交通学院学报.2005, 21(2): 70-74.
    [19]刘松,应自炉.基于局部特征和整体特征融合的面部表情识别.电子技术应用[J] 2005, 3: 4-6.
    [20] Ioannou S V,RaouzAiou A T,Tzouvaras V A, et a1. Emotion Recognition Through FacialExpression Analysis Based on a Neumfuzzy Network[J]. Neural Networks. 2005,18(4) :423-435.
    [21] P.Michel, R.E. Kaliouby. Real time facial expression recognition in video using Support Vector Machines[C]. Proceedings of the 5 International Conference on Multimodal Interfaces, Vancouver, British Columbia, Canada. 2003: 258-264.
    [22] Takahiro Otsuka, Jun Ohya. Recognition of Facial Expression Using HMM with Continuous Output Probabilities[C]. Proceedings of IEEE International Workshop on Robot and Human Communication. 1996: 323-328.
    [23]张一鸣.人脸表情识别[D].大连理工大学硕士学位论文.2006, 12: 9-10.
    [24]王冲.基于Gabor的人脸表情识别[D].昆明理工大学硕士学位论文.2008, 1: 4-5.
    [25]徐红侠.基于Gabor和局域二值模式的人脸表情识别[D].南京理工大学硕士学位论文.2008, 6: 15.
    [26] Viola P., Jones M. Rapid object detection using a boosted cascade of simple features[C]. Proceedings of IEEE Conference Computer Vision and Pattern Recognition, Kauai, Hawaii, USA, 2001: 511-518.
    [27] Lienhart R., Maydt J. An extended set of Haar-like features for rapid object detection. IEEE International Conference on Image Processing, New York, USA, 2002, 1: 900-903.
    [28]孔凡芝.基于Adaboost和支持向量机的人脸识别系统研究[D].哈尔滨工程大学工学硕士学位论文.2005, 2: 18-19.
    [29]王志良,刘芳,王莉.基于计算机视觉的表情识别技术综述[J].计算机工程.2006, 32(11): 231-233.
    [30]胡庆格.基于Gabor小波变换的人脸表情识别方法的研究[D].河北工业大学硕士学位论文.2007, 11: 18.
    [31] A.Lanitis, C.J.Taylor, T.F.Cootes. Automatic interpretation and coding of face images using fiexible models[J]. Pattern Analysis and Machine Intelligence. 1997,19 (7): 743-756.
    [32] Ying-li Tian,Lisa Brown.Real Word Real-time Automatic Recognition of Facial Expressions[C]. proceedings of IEEE workshop on performance evaluation of tracking and surveillance, Graz, Austria. 2003, 3.
    [33]武宇文.基于脸部二维形状与结构特征的表情识别研究[D].北京大学博士学位论文.2005, 5.
    [34] Xiaoyi Feng.Facial Expression Recognition Coarse-to-Fine Classification[C].IEEE 2004,Based on Local Binary Patterns and proceedings of the Fourth International Conference on Computer and Information Technology. 2004, 9(14-16): 178-183.
    [35]曹建强,童灿.基于LBP-EHMM的自动人脸表情识别[J].东南大学研究生学报.2008, 5(2): 17-21.
    [36] T. F. Cootes, C. J. Taylor. Statistical models of appearance for computer Vision[R]. Technical Report of Imaging Science and Biomedical Engineering, University of Manchester. 2000.
    [37] T.F. Cootes, C. J. Taylor, D. H. Cooper, J. Graham. Active shape models-their training and application[J]. Journal of Computer Vision and Understanding. 1995, 61(1): 38-59.
    [38]侯云舒.一种面向人脸的柔性目标理解与运动分析技术的研究[D].西北工业大学硕士论文.2003.
    [39]廉文娟,范昊,卢新民,等.使用ASM和AAM进行器官定位[J].山东科技大学学报.2002, 21(3): 40-43.
    [40]陈培俊.基于静态图像的人脸表情识别研究[D].西南交通大学硕士学位论文.2007, 4.
    [41] http://www.imm.dtu.dk/~aam/.
    [42]章毓晋.图像工程(下册):图像理解[M].北京:清华大学出版社,2007.
    [43] Haralick R.,Shanmugam K.,Distein I. Textural features for image classification[J].IEEE Transactions on Systems Man and Cybernetics. 1973, 3(6): 610-621.
    [44]王飒,郑链.基于Fisher准则和特征聚类的特征选择[J].计算机应用.2007, 27(11): 2812-2840.
    [45] Young-Suk Shin . A Neural Network Model for Classification of Facial Expressions Based on Dimension Model [C]. LNCS 3516, 2005: 941-944.
    [46]董戎萍,唐伯良.基于DCT- BP神经网络的人脸表情识别[J].微计算机信息.2005, 21(10-1):142-144.
    [47] Jingcai Fan, Hongxun Yao, Wen Gao, Yazhou Liu, Xin Liu. The Bunch-Active Shape Mode[C]. LNCS 3784, 2005:16-23.
    [48] Andreas Tewes, Rolf P.Wurtz, Christoph von der Malsburg. A Flexible Object Model for Recognition and Synthesising Facial Expression[C]. Lecture Notes in Computer Science. 2005, Vol.3546: 81-90.
    [49] Huchuan Lu, Pei Wu,Hui Lin, Deli Yang. Automatic Facial Expression Recognition[C]. Lecture Notes in Computer Science. 2006, 3972: 63-68.
    [50] P.Michel, R.E. Kaliouby. Real time facial expression recognition in video using Support Vector Machines[C]. Proceedings of the 5 International Conference on Multimodal Interfaces, Vancouver, British Columbia, Canada. 2003: 258-264.
    [51]周开利,康耀红.神经网络模型及其MATALB仿真程序设计[M].北京:清华大学出版社,2005.
    [52]周伟.基于非负矩阵稀疏分解和径向基神经网络的人脸识别方法[D].电子科技大学硕士学位论文.2006, 5.
    [53] Hansen L K, Salamon P. Neural network ensembles[J]. IEEE Trans PAMI.1990, 12(10): 993-1001.
    [54] Sollich P, Krogh A. Learning with ensembles: How over-fitting can be useful[J]. MA: MITPress, 1996: 190-196.
    [55]周艳平,夏利民,宋星光.基于Gabor滤波的多分类器集成人脸表情识别[J].长沙交通学院学报.2005, 21(2): 70-74.
    [56]王宇博,艾海舟,武勃,等.人脸表情的实时分类.计算机辅助设计与图形学学报[J].2005, 17(6): 1296-1301.

© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号

地址:北京市海淀区学院路29号 邮编:100083

电话:办公室:(+86 10)66554848;文献借阅、咨询服务、科技查新:66554700