基于相似度的人脸特征点自动定位方法的研究
|
摘要
人脸特征点的自动定位技术一直是学者们研究的热点,可以应用于三维人脸建模,人脸表情识别等领域。目前人脸特征点定位的方法较多,但在定位精度和计算速度方面都存在需要改进的地方。
本文设计了一种基于相似度的人脸特征点自动定位方法。本方法对左右眼瞳孔中心,眼角点,上下眼睑点,鼻孔外侧,鼻尖和嘴部中心、嘴角和上下嘴唇边缘点和眉毛左右边缘点以及脸部轮廓点,共29个点进行自动定位。该方法的特点是：在一定的光照干扰,胡须干扰以及人脸小角度倾斜情况下具有一定鲁棒性。且无需进行前期繁琐的手动定位制作样本集进行训练。
首先对原始图像进行了预处理,其中包括直方图均衡化和光照补偿。实验证明,预处理后的彩色图像比原图像更加清晰且亮度增加,增强了对比度,使得后续的人脸检测和特征点定位较为容易实现。
其次将预处理后的RGB图转换到YCbCr色彩空间,计算了人脸在Cb-Cr子空间中相对于肤色模型的相似度,得到人脸肤色区域亮度较高,非人脸区域亮度较低的相似度灰度图,对其均值滤波和归一化后进行二值化,分割出人脸并截取人脸部分,再对二值图做数学形态学计算去除噪声。
利用检测人脸区域时的二值图,结合先验知识和边缘检测以及唇色公式、鼻孔灰度变换和眉毛灰度变换公式定位眼睛、嘴唇、鼻子和眉毛的特征点。其中利用的鼻孔、眉毛变换公式是本课题的创新点之一。
最后,对人脸轮廓点的定位方法也是创新所在。对于额头点,可以利用两眼中心连线的中点的横坐标并结合边缘检测寻找到其纵坐标。类似地,还可定位两眼角延长线上的边缘点和两嘴角延长线上的边缘点。最后采用曲线拟合的方法定位下巴点。
通过实验验证了上述算法的可行性,实现了较准确的人脸检测与特征点定位。
Technology of automatic facial feature point localization is a hot spot of research in the field of 3D face modeling, facial expression recognition, etc. Currently, the number of facial feature point orientation method is large, but the improvement still needed on the calculation speed and detection effect of the algorithm.
This paper introduces a kind of method based on the similarity between the skin color model and test images in the Cb-Cr subspace. This method select irises center, the corner of eyes, the eyelid points, the nostril points, the tip of the nose and mouth center, corner of lips and the edge of eyebrows points and the profile points of face, a total of 29 points were automatic located. The characteristics of this method are following: In a complicated illumination, beard interference and a small angle tilt of face condition, this system which mentioned in this paper is still robust. It is not necessary to localize the facial feature points manually to train them as a sample set.
First of all, the original image is preprocessed. It includes histogram equalization and light compensation. The experiment result shows that the color image after preprocessing is clearer and brighter than the original image, and the contrast is enhanced. It makes that it is convenient for the following steps of face detection and facial feature point localization.
Second, the preprocessed images are converted from the RGB color space to the YCbCr color space. Then, the similarity between the skin color model and the preprocessed images are calculated in the Cb-Cr subspace. The similarity calculated gray-scale images make the face area brighter and the non-face region darker. After using average filter to reduce the noise points of it and normalize it, the binary images which segmented the face area can be calculated. In the binary images, the morphologic calculation is adopted for removing noise.
Third, Applying the binary images mentioned above and combining the prior-knowledge with edge detection algorithm and lip color formula, nostril gray-scale transformation function and eyebrows gray-scale transformation function to locate eyes, nose, lips and eyebrows feature points. One of the innovations in this project is the utilization of nostril gray-scale transformation function and eyebrows gray-scale transformation function.
The method of locating the six face profile points is also the innovation in this paper. Based on the y-coordinates of the corner of eyes and the corner of lips and the x-coordinate of the center of two eyes, the five points except chin point can be easily located in the edge detected images. And the chin point can be detected by applying the curve fitting algorithm and adjust the y-coordinate of it by an angle which mentioned in the following text.
At last, the experimental results verify the feasibility of this method used in this project. It is performed accurately in face detection and facial feature point localization.
本文设计了一种基于相似度的人脸特征点自动定位方法。本方法对左右眼瞳孔中心,眼角点,上下眼睑点,鼻孔外侧,鼻尖和嘴部中心、嘴角和上下嘴唇边缘点和眉毛左右边缘点以及脸部轮廓点,共29个点进行自动定位。该方法的特点是：在一定的光照干扰,胡须干扰以及人脸小角度倾斜情况下具有一定鲁棒性。且无需进行前期繁琐的手动定位制作样本集进行训练。
首先对原始图像进行了预处理,其中包括直方图均衡化和光照补偿。实验证明,预处理后的彩色图像比原图像更加清晰且亮度增加,增强了对比度,使得后续的人脸检测和特征点定位较为容易实现。
其次将预处理后的RGB图转换到YCbCr色彩空间,计算了人脸在Cb-Cr子空间中相对于肤色模型的相似度,得到人脸肤色区域亮度较高,非人脸区域亮度较低的相似度灰度图,对其均值滤波和归一化后进行二值化,分割出人脸并截取人脸部分,再对二值图做数学形态学计算去除噪声。
利用检测人脸区域时的二值图,结合先验知识和边缘检测以及唇色公式、鼻孔灰度变换和眉毛灰度变换公式定位眼睛、嘴唇、鼻子和眉毛的特征点。其中利用的鼻孔、眉毛变换公式是本课题的创新点之一。
最后,对人脸轮廓点的定位方法也是创新所在。对于额头点,可以利用两眼中心连线的中点的横坐标并结合边缘检测寻找到其纵坐标。类似地,还可定位两眼角延长线上的边缘点和两嘴角延长线上的边缘点。最后采用曲线拟合的方法定位下巴点。
通过实验验证了上述算法的可行性,实现了较准确的人脸检测与特征点定位。
Technology of automatic facial feature point localization is a hot spot of research in the field of 3D face modeling, facial expression recognition, etc. Currently, the number of facial feature point orientation method is large, but the improvement still needed on the calculation speed and detection effect of the algorithm.
This paper introduces a kind of method based on the similarity between the skin color model and test images in the Cb-Cr subspace. This method select irises center, the corner of eyes, the eyelid points, the nostril points, the tip of the nose and mouth center, corner of lips and the edge of eyebrows points and the profile points of face, a total of 29 points were automatic located. The characteristics of this method are following: In a complicated illumination, beard interference and a small angle tilt of face condition, this system which mentioned in this paper is still robust. It is not necessary to localize the facial feature points manually to train them as a sample set.
First of all, the original image is preprocessed. It includes histogram equalization and light compensation. The experiment result shows that the color image after preprocessing is clearer and brighter than the original image, and the contrast is enhanced. It makes that it is convenient for the following steps of face detection and facial feature point localization.
Second, the preprocessed images are converted from the RGB color space to the YCbCr color space. Then, the similarity between the skin color model and the preprocessed images are calculated in the Cb-Cr subspace. The similarity calculated gray-scale images make the face area brighter and the non-face region darker. After using average filter to reduce the noise points of it and normalize it, the binary images which segmented the face area can be calculated. In the binary images, the morphologic calculation is adopted for removing noise.
Third, Applying the binary images mentioned above and combining the prior-knowledge with edge detection algorithm and lip color formula, nostril gray-scale transformation function and eyebrows gray-scale transformation function to locate eyes, nose, lips and eyebrows feature points. One of the innovations in this project is the utilization of nostril gray-scale transformation function and eyebrows gray-scale transformation function.
The method of locating the six face profile points is also the innovation in this paper. Based on the y-coordinates of the corner of eyes and the corner of lips and the x-coordinate of the center of two eyes, the five points except chin point can be easily located in the edge detected images. And the chin point can be detected by applying the curve fitting algorithm and adjust the y-coordinate of it by an angle which mentioned in the following text.
At last, the experimental results verify the feasibility of this method used in this project. It is performed accurately in face detection and facial feature point localization.
引文
[1]Turk M,Pentland A, "Face Recognition Using Eigenfaces", Proc. IEEE Conf. on Computer Vision and Pattern Recognition,1999:586-591
[2]施志刚,姜彬.基于2DPCA和改进的LDA算法的人脸识别技术.第23卷第3期.2009年9月
[3]齐兴敏,刘冠梅.PCA在人脸识别方法中的比较研究.江西蓝天学院学报,第3卷第3期,2008年9月
[4]M H Yang,N Ahuja,D Kriegman. Face Detection Using Mixtures of Linear Subspaces. Automatic Face and Gesture Recognition,2000. Proceedings. Fourth IEEE International Conference on.2000.70-76.
[5]Ying-li Tian, Takeo Kanade, Jeffrey F. Cohn. Recognizing Action Units for Facial Expression Analysis. Pattern Analysis and Machine Intelligence, IEEE Transactions on,2001,23(2):97-115
[6]V Vapnik.The Nature of Statistical Learning Theory.New York,Springer-Verlag,1995.
[7]R Meir,G Raetsch.An introduction to Boosting and Leveraging.Advanced Lectures on Machine Learning,LNCS.Springer,2003.119-184.
[8]Y Freund,R E Schapire.A short introduction to boosting.Journal of Japanese society for artificial intelligence,1999,771-780.
[9]P. Viola, M. Jones, "Robust real-time object detection", ICCV Workshop on Statistical and Computation Theories of Vision,2001.
[10]Rama, A., Tarres, F., Soria-Frisch, A. "Cascade scheme face detection using a non-liniar classifier". Image Analysis for Multimedia Interactive Services.2009,: 153-156
[11]孔凡芝,张兴周,谢耀菊.基于AdaBoost的人脸检测技术.应用科技,第32卷第6期,2005年6月
[12]熊盛武,宗欣露,朱国锋.改进的基于A d a B o o s t算法的人脸检测方法.计算机应用研究.2007年第11期
[13]S Z Li,Z Q Zhang.FloatBoost learning and statistical face detection[J].IEEE Trans.Pattern Analysis and Machine Intelligence,2004,28(9):12112-1123.
[14]R Xiao,M J Li,H J Zhang.Robust multipose face detection in images[J].IEEE Trans.Circuits and Systems for Video Technology,2004,14(l):31-41.
[15]Danijela Vukadinovic and Maja Pantic, "Fully Automatic Facial Feature Point Detection Using Gabor Feature Based Boosted Classifiers", Systems, Man and Cybernetics,2005 IEEE International Conference,2005,1692-1698 Vol.2
[16]Lior W,Ian M.Robust boosting for learning from few examples [J].IEEE Conf on Computer Vision and Pattern Recognition(CVPR),2005:359-364.
[17]程丽芳,赵明昌,张向文.基于Gentleboost算法的人物检测.计算机工程与应用.2008,44(23)：187-190
[18]林志阳,张群东,张春元,易家傅.一种眼睛精确定位的人脸检测方法.海南大学学报自然科学版.第27卷第4期,2009年12月
[19]Yun Tie. LingGuan, Automatic face detection in video sequences using local normalization and optimal adaptive correlation techniques, Pattern Recognition, 2008(42),1859-1868
[20]Rein-Lien Hsu,Mohamed Abdel-Mottaleb,Anil K.Jain, "Face Detection in Color Image",IEEE Transaction on Pattern Analysis and Machine Intellegence, vol.24,No.5,May 2002
[21]杨颖,耿国华,周明全,董建民.人脸特征点自动标定方法的研究.计算机应用于软件,第26卷第3期,2009年3月
[22]邵林昌.基于混合方法的人脸检测综述.淮阳职业技术学院学报.第4期.2005年11月
[23]G.C.Feng,P.C.Yuen.Variance projection function and its application to eye detection for human face recognition.Pattern Recognition Letters.l998,19(9):899-906
[24]Zhi-Hua Zhou,Xin Geng.Projection functions for eye detection.Pattern Recognition.2004,37(5):1049-1056
[25]L.Shu-Hung,W.Shi-Lin,L.Wing-Hong.Lip image segmentation using fuzzy clustering incorporating an elliptic shape function.IEEE Transactions on Image Processing.2004,13(1):51-62
[26]张志刚,周明全,耿国华.人脸关键特征点自动标定研究.计算机工程与应用.2007,43(21)
[27]T.Kawaguchi,M.Rizon.Iris detection using intensity and edge information.Pattern Recognition.2003,36(2):549-562
[28]M.Kass,A.Witkin and D.Terzopoulos,Snakes:Active Contour Models,International Journal of Computer Vision,1987,1(4):321-331
[29]Alan L.Yuille,Peter W.Hallinan,David S.Cohen.Feature extraction from faces using deformable templates.International Journal of Computer Vision.1992,8(2):99-111
[30]A.Lanitis,C.J.Taylor and T.F.Cootes,Automatic Face Identification System Using Flexible Appearance Models,Image and Vision Computing,1995,13(5):393-401
[31]T.F.Cootes,A.Hill,C.J.Taylor and J.Haslam,Use of Active Shape Models for Locating Structures in Medical Images,Images and Vision Computing,1994,12(6)
[32]T.F.Cootes,G.Edwards and C.J.Taylor,Active Appearance Models,IEEE Trans on PAMI,2001,23(6):681-685
[33]T.F.Cootes,C.J.Taylor,D.H.Cooper,J.Graham.Active Shape Models-Their Training and Application.Computer Vision and Image Understanding.1995,61(1):38-59
[34]J.Luettin,A.N.Thacker,S.W.Beet.Locating and tracking facial speech features.In:Proceedings of ICPR'96.Vienna,Austria:1996,652-656
[35]T.F.Cootes,G.J.Edwards,C.J.Taylor.Active appearance models.Pattern Analysis and Machine Intelligence,IEEE Transactions on.2001,23(6):681-685
[36]I.Matthews,S.Baker.Active Appearance Models Revisited.International Journal of Computer Vision.2004,60(2):135-164
[37]Mohammad H. Mahoor,Mohamed Abdel-Mottaleb, "Facial Features Extraction in Color Images Using Enhanced Active Shape Model", Automatic Face and Gesture Recognition,2006. FGR2006.7th International Conference on,2006,5pp.-148
[38]刘成明,张立明.一种改进的人脸特征点定位方法.复旦学报.第45卷第4期.2006年8月
[39]S.Yan,C.Liu,S.Z.Li,H.Zhang,H.-Y.Shum,Q.Cheng.Face alignment using texture-constrained active shape models.Image and Vision Computing.2003,21(l):69-75
[40]K.-W.Wan,K.-M.Lam,K.-C.Ng.An accurate active shape model for facial feature extraction.Pattern Recognition Letters.2005,26(15):2409-2423
[41]Yan Tong,Xiaoming Liu,Frederick W. Wheeler,Peter Tu, Automatic Facial Landmark Labeling with Minimal Supervision, Computer Vision and Pattern Recognition,2009,2097-2104
[42]C.Y.Kin,R.Cipolla.A probabilistic framework for perceptual grouping of features for human face detection.In:Proceedings of Automatic Face and Gesture Recognition.Killington,U SA:1996,16-21
[43]V.Kruger,G.Sommer.Wavelet networks for face processing.Journal of Optical Society of America,2002,19(6):1112-1119
[44]R.S.Feris,J.Gemmell,K.Toyama.Hierarchical wavelet networks for facial feature localization.In:Proceedings of Automatic Face and Gesture Recognition. Washington DC,USA:2002,l 18-123
[45]MiYoung Nam, Zhan Yu, Gi Han Kim, Phill Kyu Rhee, Facial Landmark Detection System using Interest-region Model and Edge Energy Function, Systems, Man and Cybernetics,2009,2580-2584
[46]David G.Lowe.Object recognition from local scale-invariant features. International Conference on Computer Vision,Corfu,Greece,1999(9):1150-1157.
[47]David G.Lowe.Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision,60,2(2004):91-110.
[48]Dianle Zhou, Dijana Petrovska-Delacr'etaz, Bernadette Dorizzi, Automatic Landmark Location with a Combined Active Shape Model, Biometrics:Theory, Applications, and Systems,2009,1-7
[49]Zisheng LI, Jun-ichi IMAI, and Masahide KANEKO. Facial Feature Localization Using Statistical Models and SIFT Descriptors. Robot and Human Interactive Communication,2009.961-966
[50]Tie Yun, Ling Guan. Automatic Fiducial Points Detection for Facial Expressions Using Scale Invariant Feature. Multimedia Signal Processing,2009,1-6
[2]施志刚,姜彬.基于2DPCA和改进的LDA算法的人脸识别技术.第23卷第3期.2009年9月
[3]齐兴敏,刘冠梅.PCA在人脸识别方法中的比较研究.江西蓝天学院学报,第3卷第3期,2008年9月
[4]M H Yang,N Ahuja,D Kriegman. Face Detection Using Mixtures of Linear Subspaces. Automatic Face and Gesture Recognition,2000. Proceedings. Fourth IEEE International Conference on.2000.70-76.
[5]Ying-li Tian, Takeo Kanade, Jeffrey F. Cohn. Recognizing Action Units for Facial Expression Analysis. Pattern Analysis and Machine Intelligence, IEEE Transactions on,2001,23(2):97-115
[6]V Vapnik.The Nature of Statistical Learning Theory.New York,Springer-Verlag,1995.
[7]R Meir,G Raetsch.An introduction to Boosting and Leveraging.Advanced Lectures on Machine Learning,LNCS.Springer,2003.119-184.
[8]Y Freund,R E Schapire.A short introduction to boosting.Journal of Japanese society for artificial intelligence,1999,771-780.
[9]P. Viola, M. Jones, "Robust real-time object detection", ICCV Workshop on Statistical and Computation Theories of Vision,2001.
[10]Rama, A., Tarres, F., Soria-Frisch, A. "Cascade scheme face detection using a non-liniar classifier". Image Analysis for Multimedia Interactive Services.2009,: 153-156
[11]孔凡芝,张兴周,谢耀菊.基于AdaBoost的人脸检测技术.应用科技,第32卷第6期,2005年6月
[12]熊盛武,宗欣露,朱国锋.改进的基于A d a B o o s t算法的人脸检测方法.计算机应用研究.2007年第11期
[13]S Z Li,Z Q Zhang.FloatBoost learning and statistical face detection[J].IEEE Trans.Pattern Analysis and Machine Intelligence,2004,28(9):12112-1123.
[14]R Xiao,M J Li,H J Zhang.Robust multipose face detection in images[J].IEEE Trans.Circuits and Systems for Video Technology,2004,14(l):31-41.
[15]Danijela Vukadinovic and Maja Pantic, "Fully Automatic Facial Feature Point Detection Using Gabor Feature Based Boosted Classifiers", Systems, Man and Cybernetics,2005 IEEE International Conference,2005,1692-1698 Vol.2
[16]Lior W,Ian M.Robust boosting for learning from few examples [J].IEEE Conf on Computer Vision and Pattern Recognition(CVPR),2005:359-364.
[17]程丽芳,赵明昌,张向文.基于Gentleboost算法的人物检测.计算机工程与应用.2008,44(23)：187-190
[18]林志阳,张群东,张春元,易家傅.一种眼睛精确定位的人脸检测方法.海南大学学报自然科学版.第27卷第4期,2009年12月
[19]Yun Tie. LingGuan, Automatic face detection in video sequences using local normalization and optimal adaptive correlation techniques, Pattern Recognition, 2008(42),1859-1868
[20]Rein-Lien Hsu,Mohamed Abdel-Mottaleb,Anil K.Jain, "Face Detection in Color Image",IEEE Transaction on Pattern Analysis and Machine Intellegence, vol.24,No.5,May 2002
[21]杨颖,耿国华,周明全,董建民.人脸特征点自动标定方法的研究.计算机应用于软件,第26卷第3期,2009年3月
[22]邵林昌.基于混合方法的人脸检测综述.淮阳职业技术学院学报.第4期.2005年11月
[23]G.C.Feng,P.C.Yuen.Variance projection function and its application to eye detection for human face recognition.Pattern Recognition Letters.l998,19(9):899-906
[24]Zhi-Hua Zhou,Xin Geng.Projection functions for eye detection.Pattern Recognition.2004,37(5):1049-1056
[25]L.Shu-Hung,W.Shi-Lin,L.Wing-Hong.Lip image segmentation using fuzzy clustering incorporating an elliptic shape function.IEEE Transactions on Image Processing.2004,13(1):51-62
[26]张志刚,周明全,耿国华.人脸关键特征点自动标定研究.计算机工程与应用.2007,43(21)
[27]T.Kawaguchi,M.Rizon.Iris detection using intensity and edge information.Pattern Recognition.2003,36(2):549-562
[28]M.Kass,A.Witkin and D.Terzopoulos,Snakes:Active Contour Models,International Journal of Computer Vision,1987,1(4):321-331
[29]Alan L.Yuille,Peter W.Hallinan,David S.Cohen.Feature extraction from faces using deformable templates.International Journal of Computer Vision.1992,8(2):99-111
[30]A.Lanitis,C.J.Taylor and T.F.Cootes,Automatic Face Identification System Using Flexible Appearance Models,Image and Vision Computing,1995,13(5):393-401
[31]T.F.Cootes,A.Hill,C.J.Taylor and J.Haslam,Use of Active Shape Models for Locating Structures in Medical Images,Images and Vision Computing,1994,12(6)
[32]T.F.Cootes,G.Edwards and C.J.Taylor,Active Appearance Models,IEEE Trans on PAMI,2001,23(6):681-685
[33]T.F.Cootes,C.J.Taylor,D.H.Cooper,J.Graham.Active Shape Models-Their Training and Application.Computer Vision and Image Understanding.1995,61(1):38-59
[34]J.Luettin,A.N.Thacker,S.W.Beet.Locating and tracking facial speech features.In:Proceedings of ICPR'96.Vienna,Austria:1996,652-656
[35]T.F.Cootes,G.J.Edwards,C.J.Taylor.Active appearance models.Pattern Analysis and Machine Intelligence,IEEE Transactions on.2001,23(6):681-685
[36]I.Matthews,S.Baker.Active Appearance Models Revisited.International Journal of Computer Vision.2004,60(2):135-164
[37]Mohammad H. Mahoor,Mohamed Abdel-Mottaleb, "Facial Features Extraction in Color Images Using Enhanced Active Shape Model", Automatic Face and Gesture Recognition,2006. FGR2006.7th International Conference on,2006,5pp.-148
[38]刘成明,张立明.一种改进的人脸特征点定位方法.复旦学报.第45卷第4期.2006年8月
[39]S.Yan,C.Liu,S.Z.Li,H.Zhang,H.-Y.Shum,Q.Cheng.Face alignment using texture-constrained active shape models.Image and Vision Computing.2003,21(l):69-75
[40]K.-W.Wan,K.-M.Lam,K.-C.Ng.An accurate active shape model for facial feature extraction.Pattern Recognition Letters.2005,26(15):2409-2423
[41]Yan Tong,Xiaoming Liu,Frederick W. Wheeler,Peter Tu, Automatic Facial Landmark Labeling with Minimal Supervision, Computer Vision and Pattern Recognition,2009,2097-2104
[42]C.Y.Kin,R.Cipolla.A probabilistic framework for perceptual grouping of features for human face detection.In:Proceedings of Automatic Face and Gesture Recognition.Killington,U SA:1996,16-21
[43]V.Kruger,G.Sommer.Wavelet networks for face processing.Journal of Optical Society of America,2002,19(6):1112-1119
[44]R.S.Feris,J.Gemmell,K.Toyama.Hierarchical wavelet networks for facial feature localization.In:Proceedings of Automatic Face and Gesture Recognition. Washington DC,USA:2002,l 18-123
[45]MiYoung Nam, Zhan Yu, Gi Han Kim, Phill Kyu Rhee, Facial Landmark Detection System using Interest-region Model and Edge Energy Function, Systems, Man and Cybernetics,2009,2580-2584
[46]David G.Lowe.Object recognition from local scale-invariant features. International Conference on Computer Vision,Corfu,Greece,1999(9):1150-1157.
[47]David G.Lowe.Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision,60,2(2004):91-110.
[48]Dianle Zhou, Dijana Petrovska-Delacr'etaz, Bernadette Dorizzi, Automatic Landmark Location with a Combined Active Shape Model, Biometrics:Theory, Applications, and Systems,2009,1-7
[49]Zisheng LI, Jun-ichi IMAI, and Masahide KANEKO. Facial Feature Localization Using Statistical Models and SIFT Descriptors. Robot and Human Interactive Communication,2009.961-966
[50]Tie Yun, Ling Guan. Automatic Fiducial Points Detection for Facial Expressions Using Scale Invariant Feature. Multimedia Signal Processing,2009,1-6