基于Gabor小波变换与分形维的人脸情感识别
|
摘要
由于计算机在图像领域的优异性,情感计算越来越受到国内外研究者的关注,情感计算是试图使计算机能够像人类那样具有理解和表达情感能力的一个多学科交叉的新研究领域,在智能人机交互中起着非常重要的作用。人脸情感识别的研究在人工智能、计算机视觉、图像处理、模式识别、心理学等各种相关学科领域中都很有价值。
人脸情感识别是模式识别领域中一个比较活跃的研究方向。从广义上讲静态图像的人脸情感识别包括:预处理,人脸检测,情感特征提取,情感分类。本文由于采用的日本人脸库(JAFFE)是单人脸的灰度图像,所以主要从三个环节上进行了研究工作,研究内容体现在以下几个方面:
(1)首先,对人脸情感图像进行分割、消噪处理;然后对其作标准化处理,以便提取有效的特征,标准化处理包括尺度归一化和灰度均衡化。尺度归一化是将所有情感图像变换成标准尺寸的图像,并将所有图像中已标记的对应关键特征点校准到同一位置,灰度均衡化是将尺度归一化后的图像变换成同样的灰度级;最后对归一化后的图像使用固定像素的网格进一步分割情感子图像。
(2)提出了一种新颖的情感特征提取方法——Gabor小波变换与分形维数相结合。对预处理后的情感子图像的每一个网格进行Gabor小波变换以及分形维数的计算,为提高算法效率和减少特征维数,取Gabor变换后的小波系数模的均值、方差和计算所得的分形维数作为该网格的情感特征向量。
(3)通过采用Matlab中神经网络工具箱的BP、RBF神经网络,实现情感特征的多分类任务,因为使用Matlab可以方便地采用不同的核函数、激励函数。采用本文的预处理、特征提取和情感分类方法,可以实现快速鲁棒的识别工作。实验结果验证了方法的有效性。
Affective computing is more and more noticed by many researchers, because computer has an advantage in image area. Affective computing is a multidisciplinary synthetic area, which tries to enable computer to have the ability of understanding and expressing affection, just like human beings. And it plays an important role in intelligent human computer interface. Facial affective recognition is very valuable in the interrelated areas of artificial intelligence, computer vision, image management, pattern recognition, and psychology and so on.
Facial affective recognition is an active research area in pattern recognition. In broad sense, facial affective recognition of static images includes pretreatment, face detection, affective feature extraction and affective classification. In this paper, because each image only has a face and it is gray, we apply JAFFE database to making research mainly in the following aspects:
(1) Firstly, we segment each face from initial images and get rid of yawp. Then make a standardization management so as to attract effective features, which include size standardization and gray standardization. Size standardization is that all images are transformed to the standard size and the marked corresponding key feature points are also adjusted to fixed position. Gray standardization is that the gray of all images proceeded above is transformed to the same gray level. Finally, we plot out each image further by fixed pixels.
(2)We present a particular feature extraction algorithm—Gabor wavelet transformation combines with fractal dimension. For each segmented grid, we make Gabor wavelet transformation and fractal dimension compute. To improve efficiency and low the dimension, we regard the average and variance of the module from Gabor wavelet transformation along with the results from fractal dimension compute as affective feature vector of each grid.
(3)We solve the multi-classification task by using back propagation neural networks and radial basis function neural networks in Matlab, because Matlab is convenient in applying different kernel functions and incentive functions.
By using these methods of pretreatment, feature extraction algorithm and classification algorithm, the experiment can realize a fast and robust recognition task. The experiment results have testified the efficiency.
人脸情感识别是模式识别领域中一个比较活跃的研究方向。从广义上讲静态图像的人脸情感识别包括:预处理,人脸检测,情感特征提取,情感分类。本文由于采用的日本人脸库(JAFFE)是单人脸的灰度图像,所以主要从三个环节上进行了研究工作,研究内容体现在以下几个方面:
(1)首先,对人脸情感图像进行分割、消噪处理;然后对其作标准化处理,以便提取有效的特征,标准化处理包括尺度归一化和灰度均衡化。尺度归一化是将所有情感图像变换成标准尺寸的图像,并将所有图像中已标记的对应关键特征点校准到同一位置,灰度均衡化是将尺度归一化后的图像变换成同样的灰度级;最后对归一化后的图像使用固定像素的网格进一步分割情感子图像。
(2)提出了一种新颖的情感特征提取方法——Gabor小波变换与分形维数相结合。对预处理后的情感子图像的每一个网格进行Gabor小波变换以及分形维数的计算,为提高算法效率和减少特征维数,取Gabor变换后的小波系数模的均值、方差和计算所得的分形维数作为该网格的情感特征向量。
(3)通过采用Matlab中神经网络工具箱的BP、RBF神经网络,实现情感特征的多分类任务,因为使用Matlab可以方便地采用不同的核函数、激励函数。采用本文的预处理、特征提取和情感分类方法,可以实现快速鲁棒的识别工作。实验结果验证了方法的有效性。
Affective computing is more and more noticed by many researchers, because computer has an advantage in image area. Affective computing is a multidisciplinary synthetic area, which tries to enable computer to have the ability of understanding and expressing affection, just like human beings. And it plays an important role in intelligent human computer interface. Facial affective recognition is very valuable in the interrelated areas of artificial intelligence, computer vision, image management, pattern recognition, and psychology and so on.
Facial affective recognition is an active research area in pattern recognition. In broad sense, facial affective recognition of static images includes pretreatment, face detection, affective feature extraction and affective classification. In this paper, because each image only has a face and it is gray, we apply JAFFE database to making research mainly in the following aspects:
(1) Firstly, we segment each face from initial images and get rid of yawp. Then make a standardization management so as to attract effective features, which include size standardization and gray standardization. Size standardization is that all images are transformed to the standard size and the marked corresponding key feature points are also adjusted to fixed position. Gray standardization is that the gray of all images proceeded above is transformed to the same gray level. Finally, we plot out each image further by fixed pixels.
(2)We present a particular feature extraction algorithm—Gabor wavelet transformation combines with fractal dimension. For each segmented grid, we make Gabor wavelet transformation and fractal dimension compute. To improve efficiency and low the dimension, we regard the average and variance of the module from Gabor wavelet transformation along with the results from fractal dimension compute as affective feature vector of each grid.
(3)We solve the multi-classification task by using back propagation neural networks and radial basis function neural networks in Matlab, because Matlab is convenient in applying different kernel functions and incentive functions.
By using these methods of pretreatment, feature extraction algorithm and classification algorithm, the experiment can realize a fast and robust recognition task. The experiment results have testified the efficiency.
引文
[1] 何鹏,陶建华,谭铁牛.看懂人的“脸色”—人脸表情处理技术[N].计算机世界,2005-1-31
[2] Lavagetto F, Pockaj R. An Efficient Use of MPGE-4 FAP Interpolation for Facial Animation at 70bits/frame[J]. IEEE Trans. Circuits Syst. Video Techn, 2001, 11(10):1085-1097
[3] 高文,金辉.面部表情图像的分析与识别[J].计算机学报,1997,20(9):782-789
[4] 崔洁.面部表情识别方法研究[D].西安:西北工业大学,2006,1-2
[5] Fasel B, Luettin J. Automatic Facial Expression Analysis: A Survey[J]. Pattern Recognition Society, 2003, 36(1):259-275
[6] 姚锦秀.人脸检测技术的研究[D].广州:广东工业大学,2005,32-35
[7] Lyons M, Budynek J, Akamastu S. Automatic Classification of Single Facial Images[J]. IEEE Trans. Pattern Analysis and Machine Intelligence, 1999, 21(12): 1357-1362
[8] Zhang Z, Lyons M, Schuster M, et al. Comparison Between Geometry -based and Garbor-Wavelet-based Facial Expression Recognition Using Multi-layer Perceptron[C]. Proc. 3rd IEEE Int. Conf. Automatic Face and Gesture Recognition, 1998, 4:454-459
[9] Cohen I, Sebe N, Cozman F, et al. Coding, Analysis, Interpretation and Recognition of Facial Expressions[J]. Journal of Computer Vision and Image Understanding, 2003
[10] Wen Z, Huang T S. Capturing Subtle Facial Motions in 3D Face Tracking[C]. In Proceedings of IEEE International Conference on Computer Vision, Nice, France, 2003, 2:1343-1350
[11] Ford G L. Fully Automatic Coding of Basic Expressions From Video[R]. Technical Report INC-MPLab-TR-2002.03, Machine Perception Lab, Institute for Neural Computation, University of California, San Diego, CA,September 2002.
[12] Bartlett M, Braathen B, Littlewort-Ford G, et al. Automatic Analysis of Spontaneous Facial Behavior[R]: A Final Project Report. Technical Report INC-MPLab-TR-2001.08, Machine Perception Lab, Institute for Neural Computation, University of California, San Diego, 2001,12
[13] Essa I, Pentland A. Coding, Analysis, Interpretation, and Recognition of Facial Expressions[J]. IEEE Trans.on Pattern Analysis and Machine Intellengence,1997, 19(7):757-763
[14] Pantic M, Rothkrantz L. Expert System for Automatic Analysis of Facial Expression[J]. Image and Vision Computing, 2000, 18(11):881-905
[15] 金辉.人脸面部表情编码、分析、识别的研究与实现[D].哈尔滨:哈尔滨工业大学,2003
[16] Fan Y M, Cheng N, Wang Z L, et al. Real-time Facial Expression Recognition System Based on HMM and Feature Point Location[C]. LNCS 3784, 2005, 210-217
[17] Cao Y J, Zheng W M, Zhao L, et al. Expression Recognition Using Elastic Graph Matching[C]. LNCS 3784, 2005, 8-15
[18] Wang Q, Ai H Z, Xu G Y. 3D Model Based Expression Tracking in Intrinsic Expression Space[C]. Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004, 487-492
[19] Chen H, Zhang J S. Multiple Faces Detection Based on Chaos-Mutation Niche Genetic Algorithms[C]. Intelligent Control and Automation, WCICA 2004. Fifth World Congress, 2004, 5:4199-4202
[20] Pantic M, Rothkrant L J M. Facial Action Recognition for Facial Expression Analysis from Static Facial Images[J]. IEEE Trans. On System, Man, and Cybernetics, Part B, 2004, 34(3): 1449-1461
[21] 何良华,邹采荣,包永强等.人脸面部表情识别研究进展[J].电路与系统学报,2005,10(1):70-74
[22] 黄雪琳.人脸的几何特征提取与查询[D].泉州:华侨大学,2004,29-30
[23] 冈萨雷斯.数字图像处理(MATLAB 版)[M],阮秋琦(译).北京:电子工业出版社,2005
[24] 黄爱民,安向京,骆力等.数字图像处理与分析基础[M].北京:中国水利水电出版社,2005,153-154
[25] 艾竹君.基于非负矩阵分解的人脸表情识别研究[D].兰州:兰州大学,2006,24-25
[26] 章毓晋.图像处理和分析[M].北京:清华大学出版社,1999
[27] 叶敬福,詹永照.基于 Gabor 小波变换的人脸表情特征提取[J].计算机工程,2005,31(15):172-174
[28] Pantic M, Rothkrantz L J M. Automatic Analysis of Facial Expressions[J]. The State of the Art. IEEE Transactions on Pattern Analysis and Machine Intelligence,2000, 22(12):1424-1445
[29] Marian Stewart Bartlett, Martin H Lades, Terrence Sejnowski J. Independent Component Representations for Face Recognition[J]. Proceedings of the SPIE Symposium on Electronic Imaging. Human Vision and Electronic Imaging III, 1998,3299:528-539
[30] Havran C, Hupet L, Czyz J, et al. Independent Component Analysis for Face Authentication[C]. KES’2002 proceedings-knowledge-based intelligent Information and Engineering System, Crema(Italy),16-18 September 2002, 1207 -1211
[31] Chen X W, Thomas S Huang. Facial Expression Recognition: A clustering -based approach[J]. Pattern Recognition, 2003, 24(9-10):1295-1302
[32] Lyons M J, Budynek J, Akamatsu S. Automatic Classification of Single Facial Images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1999, 21(12):1357-1362
[33] Mallat S, Zhong S. Wavelets Transform Maxima Multiscale Edge in Wavelets and their Applications[C]. Boston: Jones and Bartlett, 1992, 67-104
[34] Marcelja S. Mathematical Description of the Responses of Simple Cortical Cells[J]. Journal of Optical Society of America, 1980, 70(11):1297-1300
[35] Daugman J. Uncertainty relation for resolution in space, spatial frequency and orientation optimized by two-dimensional visual cortical filters[J]. Journal of Optical Society of America, 1985, 2(7):1160-1169
[36] Kumar A. Defect Detection in Textured Materials Using Gabor Filters[J]. IEEE Transactions on Industry Applications, 2002, 38(2):425-440
[37] 王蕴红,朱勇,谭铁牛.基于虹膜识别的身份鉴别[J].自动化学报,2002, 28(1):1-10
[38] Nestares O, Navarro R, Portilla J, et al. Efficient Spatial-Domain Implementation of a Multiscale Image Representation Based on Gabor Functions[J]. Journal of Electronic Imaging, 1998, 7(1):166-173
[39] 肯尼思?法尔科.分形几何——数学基础及其应用[M].沈阳:东北大学出版社,1996
[40] Nefian A V, Hayes M H. An Embedded HMM Based Approach for Face Detection and Recognition[C]. Proceeding of IEEE Conference on Acoustics, Speech and Recognition. Phoenix, Arizona, 1999,3553-3556
[41] Chaudhuri B, Sarkar N. Texture Segmentation Using Fractal Dimensions[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1995, 17:72-77
[42] Yoo T W, Oh I S. A Fast Algorithm for Tracking Human Face Based in Chromatic Histograms[J]. Pattern Recognition, 1999, 20(10):967-978
[43] Black M J, Yacoob Y. Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion[J]. Computer Vision, 1997, 25(1):23-48
[44] 宋星光.人脸表情识别研究[D].长沙:中南大学,2006,54-55
[45] Zhao J, Kearney G. Classifying Facial Emotions by Backpropagation Neural Networks with Fuzzy Inputs[C]. Proceeding of IEEE Conference on Neural Information Processing, 1996, 1:454-457
[46] 辛威.人脸表情自动识别方法的研究[D].西安:西安理工大学,2004,54
[47] 陈 辉 , 张 家 树 .人 脸 表 情 的 混 沌 仿 生 识 别 [J].仪 器 仪 表 学 报 , 2004 ,25(4):636-637
[48] Donato G, Bartlett M S, Hager J C, et al. Classifying Facial Actions[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,1999, 21(10):974-989
[2] Lavagetto F, Pockaj R. An Efficient Use of MPGE-4 FAP Interpolation for Facial Animation at 70bits/frame[J]. IEEE Trans. Circuits Syst. Video Techn, 2001, 11(10):1085-1097
[3] 高文,金辉.面部表情图像的分析与识别[J].计算机学报,1997,20(9):782-789
[4] 崔洁.面部表情识别方法研究[D].西安:西北工业大学,2006,1-2
[5] Fasel B, Luettin J. Automatic Facial Expression Analysis: A Survey[J]. Pattern Recognition Society, 2003, 36(1):259-275
[6] 姚锦秀.人脸检测技术的研究[D].广州:广东工业大学,2005,32-35
[7] Lyons M, Budynek J, Akamastu S. Automatic Classification of Single Facial Images[J]. IEEE Trans. Pattern Analysis and Machine Intelligence, 1999, 21(12): 1357-1362
[8] Zhang Z, Lyons M, Schuster M, et al. Comparison Between Geometry -based and Garbor-Wavelet-based Facial Expression Recognition Using Multi-layer Perceptron[C]. Proc. 3rd IEEE Int. Conf. Automatic Face and Gesture Recognition, 1998, 4:454-459
[9] Cohen I, Sebe N, Cozman F, et al. Coding, Analysis, Interpretation and Recognition of Facial Expressions[J]. Journal of Computer Vision and Image Understanding, 2003
[10] Wen Z, Huang T S. Capturing Subtle Facial Motions in 3D Face Tracking[C]. In Proceedings of IEEE International Conference on Computer Vision, Nice, France, 2003, 2:1343-1350
[11] Ford G L. Fully Automatic Coding of Basic Expressions From Video[R]. Technical Report INC-MPLab-TR-2002.03, Machine Perception Lab, Institute for Neural Computation, University of California, San Diego, CA,September 2002.
[12] Bartlett M, Braathen B, Littlewort-Ford G, et al. Automatic Analysis of Spontaneous Facial Behavior[R]: A Final Project Report. Technical Report INC-MPLab-TR-2001.08, Machine Perception Lab, Institute for Neural Computation, University of California, San Diego, 2001,12
[13] Essa I, Pentland A. Coding, Analysis, Interpretation, and Recognition of Facial Expressions[J]. IEEE Trans.on Pattern Analysis and Machine Intellengence,1997, 19(7):757-763
[14] Pantic M, Rothkrantz L. Expert System for Automatic Analysis of Facial Expression[J]. Image and Vision Computing, 2000, 18(11):881-905
[15] 金辉.人脸面部表情编码、分析、识别的研究与实现[D].哈尔滨:哈尔滨工业大学,2003
[16] Fan Y M, Cheng N, Wang Z L, et al. Real-time Facial Expression Recognition System Based on HMM and Feature Point Location[C]. LNCS 3784, 2005, 210-217
[17] Cao Y J, Zheng W M, Zhao L, et al. Expression Recognition Using Elastic Graph Matching[C]. LNCS 3784, 2005, 8-15
[18] Wang Q, Ai H Z, Xu G Y. 3D Model Based Expression Tracking in Intrinsic Expression Space[C]. Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004, 487-492
[19] Chen H, Zhang J S. Multiple Faces Detection Based on Chaos-Mutation Niche Genetic Algorithms[C]. Intelligent Control and Automation, WCICA 2004. Fifth World Congress, 2004, 5:4199-4202
[20] Pantic M, Rothkrant L J M. Facial Action Recognition for Facial Expression Analysis from Static Facial Images[J]. IEEE Trans. On System, Man, and Cybernetics, Part B, 2004, 34(3): 1449-1461
[21] 何良华,邹采荣,包永强等.人脸面部表情识别研究进展[J].电路与系统学报,2005,10(1):70-74
[22] 黄雪琳.人脸的几何特征提取与查询[D].泉州:华侨大学,2004,29-30
[23] 冈萨雷斯.数字图像处理(MATLAB 版)[M],阮秋琦(译).北京:电子工业出版社,2005
[24] 黄爱民,安向京,骆力等.数字图像处理与分析基础[M].北京:中国水利水电出版社,2005,153-154
[25] 艾竹君.基于非负矩阵分解的人脸表情识别研究[D].兰州:兰州大学,2006,24-25
[26] 章毓晋.图像处理和分析[M].北京:清华大学出版社,1999
[27] 叶敬福,詹永照.基于 Gabor 小波变换的人脸表情特征提取[J].计算机工程,2005,31(15):172-174
[28] Pantic M, Rothkrantz L J M. Automatic Analysis of Facial Expressions[J]. The State of the Art. IEEE Transactions on Pattern Analysis and Machine Intelligence,2000, 22(12):1424-1445
[29] Marian Stewart Bartlett, Martin H Lades, Terrence Sejnowski J. Independent Component Representations for Face Recognition[J]. Proceedings of the SPIE Symposium on Electronic Imaging. Human Vision and Electronic Imaging III, 1998,3299:528-539
[30] Havran C, Hupet L, Czyz J, et al. Independent Component Analysis for Face Authentication[C]. KES’2002 proceedings-knowledge-based intelligent Information and Engineering System, Crema(Italy),16-18 September 2002, 1207 -1211
[31] Chen X W, Thomas S Huang. Facial Expression Recognition: A clustering -based approach[J]. Pattern Recognition, 2003, 24(9-10):1295-1302
[32] Lyons M J, Budynek J, Akamatsu S. Automatic Classification of Single Facial Images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1999, 21(12):1357-1362
[33] Mallat S, Zhong S. Wavelets Transform Maxima Multiscale Edge in Wavelets and their Applications[C]. Boston: Jones and Bartlett, 1992, 67-104
[34] Marcelja S. Mathematical Description of the Responses of Simple Cortical Cells[J]. Journal of Optical Society of America, 1980, 70(11):1297-1300
[35] Daugman J. Uncertainty relation for resolution in space, spatial frequency and orientation optimized by two-dimensional visual cortical filters[J]. Journal of Optical Society of America, 1985, 2(7):1160-1169
[36] Kumar A. Defect Detection in Textured Materials Using Gabor Filters[J]. IEEE Transactions on Industry Applications, 2002, 38(2):425-440
[37] 王蕴红,朱勇,谭铁牛.基于虹膜识别的身份鉴别[J].自动化学报,2002, 28(1):1-10
[38] Nestares O, Navarro R, Portilla J, et al. Efficient Spatial-Domain Implementation of a Multiscale Image Representation Based on Gabor Functions[J]. Journal of Electronic Imaging, 1998, 7(1):166-173
[39] 肯尼思?法尔科.分形几何——数学基础及其应用[M].沈阳:东北大学出版社,1996
[40] Nefian A V, Hayes M H. An Embedded HMM Based Approach for Face Detection and Recognition[C]. Proceeding of IEEE Conference on Acoustics, Speech and Recognition. Phoenix, Arizona, 1999,3553-3556
[41] Chaudhuri B, Sarkar N. Texture Segmentation Using Fractal Dimensions[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1995, 17:72-77
[42] Yoo T W, Oh I S. A Fast Algorithm for Tracking Human Face Based in Chromatic Histograms[J]. Pattern Recognition, 1999, 20(10):967-978
[43] Black M J, Yacoob Y. Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion[J]. Computer Vision, 1997, 25(1):23-48
[44] 宋星光.人脸表情识别研究[D].长沙:中南大学,2006,54-55
[45] Zhao J, Kearney G. Classifying Facial Emotions by Backpropagation Neural Networks with Fuzzy Inputs[C]. Proceeding of IEEE Conference on Neural Information Processing, 1996, 1:454-457
[46] 辛威.人脸表情自动识别方法的研究[D].西安:西安理工大学,2004,54
[47] 陈 辉 , 张 家 树 .人 脸 表 情 的 混 沌 仿 生 识 别 [J].仪 器 仪 表 学 报 , 2004 ,25(4):636-637
[48] Donato G, Bartlett M S, Hager J C, et al. Classifying Facial Actions[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,1999, 21(10):974-989