基于Gabor小波变换的人脸表情识别技术研究
|
摘要
表情识别正逐渐成为人工情感、人工心理、模式识别等领域的研究热点。通过运用表情识别技术使计算机具有拟人的表情识别功能,能像人一样通过表情进行交流,能通过表情读懂人的内心世界,这对构建自然和谐的人机交互环境具有重要意义。表情识别技术是情感计算的内容之一,是心理学、生理学、图像处理、机器视觉、模式识别等领域的一个极富挑战性的交叉课题。
本文首先在认真阅读国内外大量文献和书籍的基础上,对人脸表情识别课题背景、研究价值、应用领域及国内外研究现状进行了阐述。并对人脸表情识别系统所涉及到的各个步骤现今所采用的主流的方法做了比较全面的综述。这为表情识别技术的研究和探讨打下了坚实的理论基础。
其次,对一幅要进行表情识别的表情图像都进行图像预处理工作。首先对人脸检测技术进行了深入的研究,根据人脸肤色在YCbCr空间的概率分布特性和对应的肤色分布的高斯模型特征,本文采用一种基于肤色高斯模型的人脸检测算法。在YCbCr色彩空间建立高斯肤色模型,运用这一模型对肤色进行分割,然后运用人脸在几何规则上的一些先验知识对人脸区域进行定位。图像预处理的另一个工作是对图像进行归一化。归一化的内容是利用图像灰度化、旋转变换、缩放变换、灰度均衡化这四种算法,将人脸检测得到的表情图像归一化。
然后,利用Gabor小波提取表情特征的优势及弹性模板匹配方法对表情分类的优势,探索了基于Gabor小波和弹性模板匹配的表情识别方法。运用多空间尺度、多方向的Gabor滤波器组来提取表情图像的Gabor特征。并结合弹性模板匹配,计算待测表情图像与表情数据库中的表情图像的相似度,然后利用这一相似度,运用K近邻表情分类策略对表情进行分类识别。通过实验证明了该方法的可行性。
最后,运用本文所介绍的表情识别各算法,使用Visual Studio C++ 6.0开发工具在Windows XP平台上设计并实现了人脸表情识别系统。经过两个表情数据库的测试,表情识别系统的识别性能基本达到了预期要求。
Facial expression recognition gradually becomes a hot topic among artificial emotion, artificial psychology and pattern recognition. The purpose of facial expression recognition is to let the computer have the ability: communicating through facial expressions. If the computer can recognize human facial expression, it is easy to build a natural and harmonious environment for human-computer interaction. Facial expression recognition has the potential market value and wide application prospect. Facial expression recognition is one of contents of affective computing, and it is a challenging topic in psychology, physiology, image processing, machine vision, pattern recognition and other areas.
In this paper, based on the theories of digital image processing and facial recognition, this paper integrates the face detection, image preprocessing, feature extraction of facial expression, expression classification and recognition into facial expression recognition system.
Firstly, in this paper lots of literatures and books which come from home or abroad has been read during preliminary preparatory stage, on the basis of these references, this paper discuss the background, applied values and research situation about facial expression recognition. Then, this paper introduces the main existing expression recognition algorithms at home and abroad which involve in the mainstream of facial expression recognition.
Secondly, the expression images should be preprocessed before expression recognition. Studying face detection technology deeply, based on the probability distribution of skin color in YCbCr color space, this paper apply the face detection algorithm based on the Gaussian model of skin color. We build the Gaussian model of skin color in the YCbCr color space, and use the model to detect face. Another work of image preprocessing is image normalization. This work involves grayscale equalization, rotation transformation, sacling transformation and image grayscale.
Thirdly, on the basis of the advantages of Gabor wavelet and flexible template matching method on the facial expression recognition, we explore the expression recognition method based on Gabor wavelet and flexible template matching. At the same time, based on face detection, a family of Gabor filters which involves multi-scale and multi-orientation is used to extract the features of expression. And then Euclidean distance and KNN strategy are used to recognize and classify the facial expression. Experimental results demonstrate the feasibility of the method.
Finally, this paper adopt the algorithms which are described above to design and develop a facial expression recognition system. The development tool this paper use is Visual Studio C++ 6.0 development tool in the Windows XP platform. According to the experimental results on two expression databases, the performance of facial expression recognition system has basically reached the expected requirements.
本文首先在认真阅读国内外大量文献和书籍的基础上,对人脸表情识别课题背景、研究价值、应用领域及国内外研究现状进行了阐述。并对人脸表情识别系统所涉及到的各个步骤现今所采用的主流的方法做了比较全面的综述。这为表情识别技术的研究和探讨打下了坚实的理论基础。
其次,对一幅要进行表情识别的表情图像都进行图像预处理工作。首先对人脸检测技术进行了深入的研究,根据人脸肤色在YCbCr空间的概率分布特性和对应的肤色分布的高斯模型特征,本文采用一种基于肤色高斯模型的人脸检测算法。在YCbCr色彩空间建立高斯肤色模型,运用这一模型对肤色进行分割,然后运用人脸在几何规则上的一些先验知识对人脸区域进行定位。图像预处理的另一个工作是对图像进行归一化。归一化的内容是利用图像灰度化、旋转变换、缩放变换、灰度均衡化这四种算法,将人脸检测得到的表情图像归一化。
然后,利用Gabor小波提取表情特征的优势及弹性模板匹配方法对表情分类的优势,探索了基于Gabor小波和弹性模板匹配的表情识别方法。运用多空间尺度、多方向的Gabor滤波器组来提取表情图像的Gabor特征。并结合弹性模板匹配,计算待测表情图像与表情数据库中的表情图像的相似度,然后利用这一相似度,运用K近邻表情分类策略对表情进行分类识别。通过实验证明了该方法的可行性。
最后,运用本文所介绍的表情识别各算法,使用Visual Studio C++ 6.0开发工具在Windows XP平台上设计并实现了人脸表情识别系统。经过两个表情数据库的测试,表情识别系统的识别性能基本达到了预期要求。
Facial expression recognition gradually becomes a hot topic among artificial emotion, artificial psychology and pattern recognition. The purpose of facial expression recognition is to let the computer have the ability: communicating through facial expressions. If the computer can recognize human facial expression, it is easy to build a natural and harmonious environment for human-computer interaction. Facial expression recognition has the potential market value and wide application prospect. Facial expression recognition is one of contents of affective computing, and it is a challenging topic in psychology, physiology, image processing, machine vision, pattern recognition and other areas.
In this paper, based on the theories of digital image processing and facial recognition, this paper integrates the face detection, image preprocessing, feature extraction of facial expression, expression classification and recognition into facial expression recognition system.
Firstly, in this paper lots of literatures and books which come from home or abroad has been read during preliminary preparatory stage, on the basis of these references, this paper discuss the background, applied values and research situation about facial expression recognition. Then, this paper introduces the main existing expression recognition algorithms at home and abroad which involve in the mainstream of facial expression recognition.
Secondly, the expression images should be preprocessed before expression recognition. Studying face detection technology deeply, based on the probability distribution of skin color in YCbCr color space, this paper apply the face detection algorithm based on the Gaussian model of skin color. We build the Gaussian model of skin color in the YCbCr color space, and use the model to detect face. Another work of image preprocessing is image normalization. This work involves grayscale equalization, rotation transformation, sacling transformation and image grayscale.
Thirdly, on the basis of the advantages of Gabor wavelet and flexible template matching method on the facial expression recognition, we explore the expression recognition method based on Gabor wavelet and flexible template matching. At the same time, based on face detection, a family of Gabor filters which involves multi-scale and multi-orientation is used to extract the features of expression. And then Euclidean distance and KNN strategy are used to recognize and classify the facial expression. Experimental results demonstrate the feasibility of the method.
Finally, this paper adopt the algorithms which are described above to design and develop a facial expression recognition system. The development tool this paper use is Visual Studio C++ 6.0 development tool in the Windows XP platform. According to the experimental results on two expression databases, the performance of facial expression recognition system has basically reached the expected requirements.
引文
[1]张利伟,张航,张玉英.面部表情识别方法综述[J].自动化技术与应用,2009,Vol.1:93-97.
[2]王济军,阴山燕.现代远程教育的情感缺失及其补偿策略[J].中国远程教育, 2005.3.
[3] Ekman P, Friesen W V. Facial Action Coding System [M].Consulting Psychologists Press, 1978.
[4]刘晓旻,谭华春,章毓晋.人脸表情识别研究新进展[J].中国图象图形学报, 2006,Vol.10 : 1359-1368.
[5] Michel F V, Maja P, Zara A, et al. Spontaneous vs Posed Facial Behavior:automatic analysis of brow actions [J]. Proceedings of the 8PthP International Conference on Mutimodal Interfaces, 2006, vol.1: 162-170.
[6] Tong Y, Wenhui L, Qiang J, et al. Facial Action Unit Recognition by Exploiting their Dynamic and Senmantic Relationships [J].IEEE Transaction on Pattern Analysis and Machine Intelligence, 2007, Vol.29: 1683-1699.
[7] Jun M, Baocai Y, Kongqiao W, et al. A hierarchical Multiscale and Multiangle System for Human Face Detection in a Complex Background Using Gravity-center Template [J]. Pattern Recognition, 1999, Vol.32(7): 1237-1248.
[8] Pantic M, Rothkrantz L. Facial Action Recognition for Facial Expression Analysis from Static Face Images [C]. IEEE Transactions on Systems, Man, and Cybernetics,2004, Vol.34(3): 1449-1461.
[9]齐永锋,火元莲.一种基于肤色的人脸检测与定位方法[J].计算机应用, 2009, Vol.3(46): 34-37.
[10]吴桂云,武妍.一种基于KL变换和镶嵌图模型的人脸检测方法[J].计算机工程, 2004, Vol.21: 120-122.
[11]魏诗国.素描[M].北京:高等教育出版社, 2000.
[12] Takeo Kanade. Picture Processing System by Computer Complex and Recognition of Human Faces[J]. Kyoto: Kyoto University, 1973.11.
[13] Sirohey S A. Human Face Segmentation and Identification[R]. Technical Report CS-TR-3176, University of Maryland, 1993.
[14] Chetverikov D, Lerch A. Multiresolution Face Detection [C]. Theoretical Foundations of Computer Vision, 1993, vol.69: 131-140.
[15] Graf H P, Cosatto E, Gibbon D, et al. Multimodal System for Locating Heads and Faces [C]. The Second International Conference on Automatic Face and Gesture Recognition, 2003: 88-93.
[16] Lee C H, Kim J S, Park K H. Automatic Human Face Location in a Complex Background Using Motion and Color Information [J]. Pattern Recognition, 2003, Vol.29(11): 1877-1889.
[17] Kapfer M, Benois-Pineau J. Detection of Human Faces in Color Image Sequences with Arbitrary Motions for Very Low Bitrate Videophone Coding[J]. Pattern Recognition Letters, 1999, Vol.18(14): 1503-1518.
[18] Dai Y, Nakano Y. Face-Texture Model Based on SGLD and Its Application in Face Detection in a Color Sence[J]. Pattern Recognition, 2003, Vol.29(6): 1007-1017.
[19] Yang M H, Ahuja N. Detecting Human Faces in Color Images [C]. Proceedings of the InternationalConference on Imaging Processing, 2000, Vol.1: 127-130.
[20] Schwerd K, Crowlet J L. Robust Face Tracking Using Color [C]. Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition, 2006: 90-95.
[21] Craw I, Ellis H, Lishman J. Finding Face Features [C]. Proceedings Second European Conf. Computer Vision, 1996: 92-96.
[22] Legung T K, Burl M C, Pweona P. Finding Faces in Cluttered Scenes Using Random Labeled Graph Matching[C]. Fifth IEEE Conference Computer vision,1998: 637-644.
[23] Yuille A, Hallinan P, Cohen D. Feature Extraction from Faces Using Deformable Templates [J]. International Journal of Computer Vision, 1998, Vol.8(2): 99-111.
[24] Lades M, Vorgrubben J C. Distortion Invariant Object Recognition in the Dynamic Link Architecture [J]. IEEE Trans, On Computer, 1996, Vol.42(3): 300-310.
[25] Nastar C , et al. Computer Vision and Image Understanding[M]. New York: Thomson Reuters, 1997: 179-191.
[26] Kass M, Witkin A, Terzopoulos D. Snakes: Active Contour Models [J]. Int. J. Computer Vision, 1988, Vol.1(4): 321-331.
[27] Cootes T F, Taylor C J. Locating Faces using Statistical Feature Detectors [C]. Proceedings of Second Int’l Conf, Automatic Face and Gesture Recognition, 1996: 204-209.
[28] Cootes T F, Taylor C J. Active Shape Models-smart Snakes[C]. Proceedings of British Machine Vision Conference, 1998: 226-275.
[29] Lanitis A, Taylor C J, Cootes T F. An Automatic Face Indentification System using Flexible Appearance Models[J]. Image and Vision Computing, 1999, Vol.13(5) : 393-401.
[30] Lanitis A, Taylor C J, Cootes T F. Automatic Tracking, Coding and Reconstruction of Human Faces, Using flexible appearance models [J]. IEEE Electronic Letters, 2000, Vol.30: 1578-1579.
[31] M. Ben ammar, M. Neji. Affective e-Learning Framework[J]. Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education Chesapeake, 2006: 1595-1602.
[32]吴朋,周激流等.基于特征融合的面部表情识别[J].四川大学学报(自然科学版), 2009, Vol.46(3): 583-588.
[33] Kim K I, Jung K, Kim H J. Face Recognition Using Kernel Principal Component Analysis [J]. IEEE Signal Processing Letters, 2002, Vol.9(2): 40-42.
[34] Yuen P C, Lai J H. Face Representation Using Independent Component Analysis for Face Recognition[J]. Pattern Recognition, 2002, Vol.35(6): 1247-1257.
[35] Beak K, Draper B A, Beveridge J R. PCA vs ICA: A Comparison[OL]. http: //www.dodcounterdrug.com/facial recognition/FRVT2004/frvt2004.htm.
[36] Essa I A, Pentland A P. Dynamic of Facial Expression: Recognition of Facial Actions,IEEE Transactions on System, Man and Cybernetics ,2006, Vol.19(7) : 757-763.
[37] Terzopoulos D, Watwes K. Real-time expression cloning using appearance models [J].Mutimodal Interfaces and Machine Learning for Mutimodal Interaction, 2007, Vol.15(6) : 569-579.
[38]郭秀梅,刘贤喜.基于肤色的人脸检测算法的研究[C].山东农业大学硕士学位论文, 2008.
[39]徐翠,李然.基于YCbCr空间的高斯肤色模型的人脸检测[J]..软件导刊, 2009.2,Vol.8(2):179-181.
[40]龚杨璟,吴薇.一种基于肤色的人脸检测方法[J].中国新科技新产品, 2010, Vol.23: 37.
[41] Erik H, Boon K L. Face Detection of A Survey [J]. Computer Vision and Image Understanding, 2001, Vol.83: 236-274.
[42] Rafael C G, Richard E W.数字图像处理(第二版)[M]. 2007.8.
[43]张静远,张冰.基于小波变换的特征提取方法分析[J].信号处理, 2000, Vol.16(2): 156-162.
[44] Gabor D. Theory of Communication[J] Journal of the Institute of Electrical Engineers, 1946, Vol.93(26) : 429-457.
[45] Lee T S. Image Representation using 2D Gabor wavelets [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1996, Vol.18(10) : 959-971.
[46]高晓兴,王文佳,李仁睦等. Gabor小波与表情组合模板相结合的表情识别[J].微计算机信息, 2010, Vol.26(2-1) : 10-12.
[47] Phillips P J, Moon H, Rizvi S A, et al. The FERET Evaluation Methodology for Face-recognition Algorithms[J]. IEEE Transaction on Pattern Analysis and Machine Intelligence, 2000, Vol.22(10) : 1090-1104.
[48] Pardas M, Bonafonte A. Facial Expression Recognition using Embedded Hidden Markov Model [J]. The Proceedings of the 2009 IEEE International Conference on Systems, Man and Cybernetics, 2009:1568-1572.
[49] Sayeed M S, Prabir B. Classification of Facial Expressions Using K-Nearest Neighbor Classifier[J]. Lecture Notes in Computer Science, 2007, Vol.4418: 555-566.
[2]王济军,阴山燕.现代远程教育的情感缺失及其补偿策略[J].中国远程教育, 2005.3.
[3] Ekman P, Friesen W V. Facial Action Coding System [M].Consulting Psychologists Press, 1978.
[4]刘晓旻,谭华春,章毓晋.人脸表情识别研究新进展[J].中国图象图形学报, 2006,Vol.10 : 1359-1368.
[5] Michel F V, Maja P, Zara A, et al. Spontaneous vs Posed Facial Behavior:automatic analysis of brow actions [J]. Proceedings of the 8PthP International Conference on Mutimodal Interfaces, 2006, vol.1: 162-170.
[6] Tong Y, Wenhui L, Qiang J, et al. Facial Action Unit Recognition by Exploiting their Dynamic and Senmantic Relationships [J].IEEE Transaction on Pattern Analysis and Machine Intelligence, 2007, Vol.29: 1683-1699.
[7] Jun M, Baocai Y, Kongqiao W, et al. A hierarchical Multiscale and Multiangle System for Human Face Detection in a Complex Background Using Gravity-center Template [J]. Pattern Recognition, 1999, Vol.32(7): 1237-1248.
[8] Pantic M, Rothkrantz L. Facial Action Recognition for Facial Expression Analysis from Static Face Images [C]. IEEE Transactions on Systems, Man, and Cybernetics,2004, Vol.34(3): 1449-1461.
[9]齐永锋,火元莲.一种基于肤色的人脸检测与定位方法[J].计算机应用, 2009, Vol.3(46): 34-37.
[10]吴桂云,武妍.一种基于KL变换和镶嵌图模型的人脸检测方法[J].计算机工程, 2004, Vol.21: 120-122.
[11]魏诗国.素描[M].北京:高等教育出版社, 2000.
[12] Takeo Kanade. Picture Processing System by Computer Complex and Recognition of Human Faces[J]. Kyoto: Kyoto University, 1973.11.
[13] Sirohey S A. Human Face Segmentation and Identification[R]. Technical Report CS-TR-3176, University of Maryland, 1993.
[14] Chetverikov D, Lerch A. Multiresolution Face Detection [C]. Theoretical Foundations of Computer Vision, 1993, vol.69: 131-140.
[15] Graf H P, Cosatto E, Gibbon D, et al. Multimodal System for Locating Heads and Faces [C]. The Second International Conference on Automatic Face and Gesture Recognition, 2003: 88-93.
[16] Lee C H, Kim J S, Park K H. Automatic Human Face Location in a Complex Background Using Motion and Color Information [J]. Pattern Recognition, 2003, Vol.29(11): 1877-1889.
[17] Kapfer M, Benois-Pineau J. Detection of Human Faces in Color Image Sequences with Arbitrary Motions for Very Low Bitrate Videophone Coding[J]. Pattern Recognition Letters, 1999, Vol.18(14): 1503-1518.
[18] Dai Y, Nakano Y. Face-Texture Model Based on SGLD and Its Application in Face Detection in a Color Sence[J]. Pattern Recognition, 2003, Vol.29(6): 1007-1017.
[19] Yang M H, Ahuja N. Detecting Human Faces in Color Images [C]. Proceedings of the InternationalConference on Imaging Processing, 2000, Vol.1: 127-130.
[20] Schwerd K, Crowlet J L. Robust Face Tracking Using Color [C]. Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition, 2006: 90-95.
[21] Craw I, Ellis H, Lishman J. Finding Face Features [C]. Proceedings Second European Conf. Computer Vision, 1996: 92-96.
[22] Legung T K, Burl M C, Pweona P. Finding Faces in Cluttered Scenes Using Random Labeled Graph Matching[C]. Fifth IEEE Conference Computer vision,1998: 637-644.
[23] Yuille A, Hallinan P, Cohen D. Feature Extraction from Faces Using Deformable Templates [J]. International Journal of Computer Vision, 1998, Vol.8(2): 99-111.
[24] Lades M, Vorgrubben J C. Distortion Invariant Object Recognition in the Dynamic Link Architecture [J]. IEEE Trans, On Computer, 1996, Vol.42(3): 300-310.
[25] Nastar C , et al. Computer Vision and Image Understanding[M]. New York: Thomson Reuters, 1997: 179-191.
[26] Kass M, Witkin A, Terzopoulos D. Snakes: Active Contour Models [J]. Int. J. Computer Vision, 1988, Vol.1(4): 321-331.
[27] Cootes T F, Taylor C J. Locating Faces using Statistical Feature Detectors [C]. Proceedings of Second Int’l Conf, Automatic Face and Gesture Recognition, 1996: 204-209.
[28] Cootes T F, Taylor C J. Active Shape Models-smart Snakes[C]. Proceedings of British Machine Vision Conference, 1998: 226-275.
[29] Lanitis A, Taylor C J, Cootes T F. An Automatic Face Indentification System using Flexible Appearance Models[J]. Image and Vision Computing, 1999, Vol.13(5) : 393-401.
[30] Lanitis A, Taylor C J, Cootes T F. Automatic Tracking, Coding and Reconstruction of Human Faces, Using flexible appearance models [J]. IEEE Electronic Letters, 2000, Vol.30: 1578-1579.
[31] M. Ben ammar, M. Neji. Affective e-Learning Framework[J]. Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education Chesapeake, 2006: 1595-1602.
[32]吴朋,周激流等.基于特征融合的面部表情识别[J].四川大学学报(自然科学版), 2009, Vol.46(3): 583-588.
[33] Kim K I, Jung K, Kim H J. Face Recognition Using Kernel Principal Component Analysis [J]. IEEE Signal Processing Letters, 2002, Vol.9(2): 40-42.
[34] Yuen P C, Lai J H. Face Representation Using Independent Component Analysis for Face Recognition[J]. Pattern Recognition, 2002, Vol.35(6): 1247-1257.
[35] Beak K, Draper B A, Beveridge J R. PCA vs ICA: A Comparison[OL]. http: //www.dodcounterdrug.com/facial recognition/FRVT2004/frvt2004.htm.
[36] Essa I A, Pentland A P. Dynamic of Facial Expression: Recognition of Facial Actions,IEEE Transactions on System, Man and Cybernetics ,2006, Vol.19(7) : 757-763.
[37] Terzopoulos D, Watwes K. Real-time expression cloning using appearance models [J].Mutimodal Interfaces and Machine Learning for Mutimodal Interaction, 2007, Vol.15(6) : 569-579.
[38]郭秀梅,刘贤喜.基于肤色的人脸检测算法的研究[C].山东农业大学硕士学位论文, 2008.
[39]徐翠,李然.基于YCbCr空间的高斯肤色模型的人脸检测[J]..软件导刊, 2009.2,Vol.8(2):179-181.
[40]龚杨璟,吴薇.一种基于肤色的人脸检测方法[J].中国新科技新产品, 2010, Vol.23: 37.
[41] Erik H, Boon K L. Face Detection of A Survey [J]. Computer Vision and Image Understanding, 2001, Vol.83: 236-274.
[42] Rafael C G, Richard E W.数字图像处理(第二版)[M]. 2007.8.
[43]张静远,张冰.基于小波变换的特征提取方法分析[J].信号处理, 2000, Vol.16(2): 156-162.
[44] Gabor D. Theory of Communication[J] Journal of the Institute of Electrical Engineers, 1946, Vol.93(26) : 429-457.
[45] Lee T S. Image Representation using 2D Gabor wavelets [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1996, Vol.18(10) : 959-971.
[46]高晓兴,王文佳,李仁睦等. Gabor小波与表情组合模板相结合的表情识别[J].微计算机信息, 2010, Vol.26(2-1) : 10-12.
[47] Phillips P J, Moon H, Rizvi S A, et al. The FERET Evaluation Methodology for Face-recognition Algorithms[J]. IEEE Transaction on Pattern Analysis and Machine Intelligence, 2000, Vol.22(10) : 1090-1104.
[48] Pardas M, Bonafonte A. Facial Expression Recognition using Embedded Hidden Markov Model [J]. The Proceedings of the 2009 IEEE International Conference on Systems, Man and Cybernetics, 2009:1568-1572.
[49] Sayeed M S, Prabir B. Classification of Facial Expressions Using K-Nearest Neighbor Classifier[J]. Lecture Notes in Computer Science, 2007, Vol.4418: 555-566.