面向人脸合成的脸部特征提取
|
摘要
脸部特征提取在人脸识别、人脸表情分析与合成、人脸变形和三维虚拟人脸
合成等方面具有非常重要的应用，因而成为国内外的一个研究热点。但由于脸部
特征的复杂性，以及人脸图像中环境和光照的变化使得它成为一个比较困难的研
究问题，特别是当人脸图像具有较大旋转角度时更增加了这一问题的复杂性。
本文在总结前人各种特征提取方法的基础上，实现了一个由粗到细、渐进的
人脸特征提取系统，能够提取人脸图像中的眼睛和嘴巴精确特征。本系统主要包
括两个新提出的算法，即人脸姿态估计策略和三维可变形模板匹配技术。
一般的人脸姿态估计方法要通过提取脸部特征点之间的关系或根据已知的
三维人脸模型来估测人脸的方向。本文提出了一种基于头发模板的人脸姿态估计
策略，能够直接从人脸图像中估算出人脸的大致方向，并且经过进一步矫正，得
到更加精确的结果。
为了能够提取多方向人脸图像中的器官精确位置，本文提出了一个参数化的
三维可变形模板匹配技术，并以眼睛和嘴巴为例子定义了三维眼睛可变形模板和
三维嘴巴可变形模板。这些三维可变形模板与人脸图像进行匹配时，还将分别映
射成二维映射模板，并分别定义了眼睛映射模板和嘴巴映射模板的能量函数，最
后通过贪心算法求取能量函数的最小值，从而得到眼睛和嘴巴的精确特征。
Human facial features extraction has become one of the important issues in lots of significant applications, such as face recognition, facial expression analysis and synthesis, face morphing and virtual 3D face synthesis. To extract the exact features of human face is quiet a hard nun to break, mainly for the complicated structure of thce, as well as thc unknown light condition, and it would become more difficult for the face with lager vertical rotation.
Here a gradual feature extraction method has been proposed to extract the exact features of eyes and mouth, in which there are two new techniques. namely pose estimation based on hair templates and 3D deformable template matching.
A new pose estimation strategy based on seven hair templates has been presented to estimate the orientation of face. The face orientation can be obtained directly from the face image using the new pose estimation, which is different from the traditional method according to the relations between feature points or the reconstructed 3D face model.
A new 3D deforrnable template matching technique is also presented here to extract the exact features of eyes and mouth. When the 3D deformable template is applied to match the face image, it should be mapped into 2D template and the energy functions of 2D eye template and 2D mouth template should also be defined respectively. At last the Greedy Algorithm should be employed to minimize the value of energy functions, and the facial features could then be obtained.
合成等方面具有非常重要的应用，因而成为国内外的一个研究热点。但由于脸部
特征的复杂性，以及人脸图像中环境和光照的变化使得它成为一个比较困难的研
究问题，特别是当人脸图像具有较大旋转角度时更增加了这一问题的复杂性。
本文在总结前人各种特征提取方法的基础上，实现了一个由粗到细、渐进的
人脸特征提取系统，能够提取人脸图像中的眼睛和嘴巴精确特征。本系统主要包
括两个新提出的算法，即人脸姿态估计策略和三维可变形模板匹配技术。
一般的人脸姿态估计方法要通过提取脸部特征点之间的关系或根据已知的
三维人脸模型来估测人脸的方向。本文提出了一种基于头发模板的人脸姿态估计
策略，能够直接从人脸图像中估算出人脸的大致方向，并且经过进一步矫正，得
到更加精确的结果。
为了能够提取多方向人脸图像中的器官精确位置，本文提出了一个参数化的
三维可变形模板匹配技术，并以眼睛和嘴巴为例子定义了三维眼睛可变形模板和
三维嘴巴可变形模板。这些三维可变形模板与人脸图像进行匹配时，还将分别映
射成二维映射模板，并分别定义了眼睛映射模板和嘴巴映射模板的能量函数，最
后通过贪心算法求取能量函数的最小值，从而得到眼睛和嘴巴的精确特征。
Human facial features extraction has become one of the important issues in lots of significant applications, such as face recognition, facial expression analysis and synthesis, face morphing and virtual 3D face synthesis. To extract the exact features of human face is quiet a hard nun to break, mainly for the complicated structure of thce, as well as thc unknown light condition, and it would become more difficult for the face with lager vertical rotation.
Here a gradual feature extraction method has been proposed to extract the exact features of eyes and mouth, in which there are two new techniques. namely pose estimation based on hair templates and 3D deformable template matching.
A new pose estimation strategy based on seven hair templates has been presented to estimate the orientation of face. The face orientation can be obtained directly from the face image using the new pose estimation, which is different from the traditional method according to the relations between feature points or the reconstructed 3D face model.
A new 3D deforrnable template matching technique is also presented here to extract the exact features of eyes and mouth. When the 3D deformable template is applied to match the face image, it should be mapped into 2D template and the energy functions of 2D eye template and 2D mouth template should also be defined respectively. At last the Greedy Algorithm should be employed to minimize the value of energy functions, and the facial features could then be obtained.
引文
[1] F. Pighin, J. Meeker, D. Lischinski, R. Szeliski, D. H. Salesin, Synthesizing Realistic Facial Expressions from Photographs, Siggraph proceedings, 1998, pp. 75-84
[2] Frederic Pighin,Joel Auslander,Dani Lischinski, David H.Salesin,Richard Szeliski. Realistic Facial Animation Using Image-Based 3D Morphing. TR UW-CSE-97-01-03, 1997．
[3] Richard Szeliski and Sing Bing Kang, Recovering 3D Shape and Motion from Image Streams using Non-Linear Least Squares, Digital Equipment Corporation, Cambridge Research Lab, Technical Report, March, 1993．
[4] Volker Blanz,Thomas Vetler. A Morphable Model for The Synthesis of 3D Facial. SIGGRAPH99, 1999．
[5] 吴思宁,具有真实感的特定人脸模型合成方法的研究,北京工业大学硕士毕业论文 1997
[6] F.I. Parke. Computer generated animation of faces. Proceedings ACM annual conference., August, 1972．
[7] Horace M.S.Ip and Lijun,Yin. Constructing a 3D Individualized Head Model from Two Orthogonal Views. The Visual Computer( 1996) 12:254-266．
[8] Frederic I.Parke and Keith Waters. Computer Facial Antimation, A.K Peters. Wellesley, Massachusetts, 1996．
[9] Takaaki Akimoto, Yasuhito Suenaga, and Richard S. Wallace, Automatic Creation of 3D Facial Models. IEEE Computer Graphics & Applications, Sep., 1993．
[10] Y.C. Lee, D. Terzopoulos, and Keith Waters. Constructing physics-based fa-cial models of individuals. Visual Computer, Proceedings of Graphics Interface 93:1-8, 1993
[11] Yuencheng Lee. Demetri Terzopoulos, and Keith Waters, Realistic Modeling for Facial Animation, In Computer Graphics (Proc. SIGGRAPH), pp. 55-62, 1996．
[12] Lee W. S., Kalra P., Magnenat-Thalmann N, "Model Based Face Reconstruction for Animation", In Proc. Multimedia Modeling (MMM'97) , Singapore, pp. 323-338,1997．
[13] D. DeCarlos, D. Metaxas, and M. Stone. An anthropometric face model us-ing variational techniques. In Computer Graphics Proceedings SIGGRAPH'98,
[14] B. Guenter, C. Grimm, D. Wolf, H. Malvar, and F. Pighin. Making faces. In Computer Graphics Proceedings SIGGRAPH'98, pages 55-66, 1998
[15] Z.liu, Z.Zhang,C.Jacobs and M.Cohen. Rapid Modeling of Animated Faces From Video. The Thirt International Conference on Visual Computing, Page: 58-67． Sep.2000．
[16] P. Fua and C. Miccio, Animated Heads from Ordinary Images: A Least Squares Approach, Computer Vision and Image Understanding, 1999, In Press.
[17] T. Cootes, C. Taylor, D. Cooper, J. Graham, Active Shape Models-Their Training and Application. Computer Vision and Image Understanding, January 1995, Vol. 61, No. 1, pp.38-59．
[18] T.F. Cootes, G.J. Edwards, and C.J. Taylor. Active appearance models. In Burkhardt and Neumann, editors. Computer Vision-ECCV'98 Vol. II, Freiburg, Germany, 1998． Springer, Lecture Notes in Computer Science 1407．
[19] A. Lanitis, C. J. Taylor, and T. F. Cootes. Automatic interpretation and coding of face images using flexible models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7) :743-756,1997．
[20] J. Miao, W.Gao, Y.Q.Chen, et al, Gravity-Center Template Based Human Face Feature Detection, Advances in Multimodal Interfaces-ICMI 2000, pp207-214．
[21] M. Kass, A. Witkin, and D. Terzopoulos. "Snakes: Active Contour Models", Internation Journal of Computer Vision, pp: 321-331, 1988．
[22] Williams, Donna and Shah, Mubarak. "A Fast Algorithm for Active Contours and Curvature Estimation", CVGIP: Image Understanding. Vol. 55, No. 1, January 1992． pp. 14-26．
[23] Stever.Gunn and Marks.Nixon, Global and local active contour for head boundary extraction, International Journal of Computer Vision, 30(1) , pp:43-54,1998．
[24] B. Li and N. Roeder, Face contour extraction from front-view images, Pattern recognition, 28(8) , 1167-1179, 1995．
[25] H.A. Rowley, S. Baluja, and T. Kanade, Neural network-based face detection, IEEE-PAMI, 20(1) , 23-38, 1998．
[26] Paul Debevec. A Neural Network for Facial Feature Location. UC Berkeley CS283 Project Report, December 1992． http://www.debevec.org/face recognition.html
[27] A.L. Yuille, Peter W.Hallinan and David S.Cohen, Feature extraction from faces using deformable templates, International Journal of Computer Vision, 8(2) : 99-111, 1992．
[28] J.Y. Deng and F. Lai, Region-based template deformation and masking for eye-feature
extraction and description, Pattern recognition, 30(3) , 403-419, 1997．
[29] S.H. Jeng, H.Y.M. Liao, et al, Facial feature detection using geometrical face model: an efficient approach, Pattern recognition, 31(3) , 273-282, 1998．
[30] K. Lam and H. Yan, Locating and extracting the eye in human face images, Pattern recognition, 29(5) . 771-779, 1996．
[31] G.Chow and X.Li, Towards a system for automatic facial feature detection, PR 26(12 ), 1739-1755, 1993．
[32] X.Jia and M.S.Nixon, Analysing front view face profile for face recognition via the Walsh transform. PRL 15, 551-558, 1994．
[33] X.Song. C.W.Lce and S.Tsuji, Extracting facial features with partial feature template, Proc. ACCV, 751-754, 1994．
[34] T.C.Chang, T.S.Huang and C.Novak, Facial feature extraction from color images, Proc. IJCPR, 39-43, 1994．
[35] X.Xie, R.Sudhakar and H.Zhuang, On improving eye feature extraction using deformable templates, Pattern recognition, 27(6) ,pp: 791-799,1994．
[36] Kin-Man.Lam and Hong Yan, Location and extracting the eye in human face images, Pattern recognition. 29(5) .771-779,1996．
[37] D. Jang and H.Choi, Active models for tracking moving objects, Pattern recognition, 33(2000) ,1135-1146,2000．
[38] T.F.Cootes,C.J.Taylor,D.H.Cooper And J.Graham, Active Shape Models-Their Training and Application. Computer Vision Graphics and Image Understanding, Vol61,No l.pp 38-59,1995．
[39] A.Ianitis.C.J.Taylor.T.F.Cootes and T.Ahmed. Automatic Interpretation of Human Faces and Hand Gestures Using Flexible Temolates. To appear in the proceedings of the International Workshop on Face and Gesture Recognition, 1995．
[40] Shimizu.l, Zhang.Z, Akamatsu.S etc. Head pose determination from one image using a generic model. 3rd IEEE International conference on automatic face and gesture recognition, pp.100-105, Nara, Japan, April. 1998．
[41] Volker Kr(?)ger, Sven Brims and Gerald Sommer Efficient Head Pose Estimation with Gabor Wavelet Networks BMVC, Bristol, Great Britain, 11-14 September 2000．
[2] Frederic Pighin,Joel Auslander,Dani Lischinski, David H.Salesin,Richard Szeliski. Realistic Facial Animation Using Image-Based 3D Morphing. TR UW-CSE-97-01-03, 1997．
[3] Richard Szeliski and Sing Bing Kang, Recovering 3D Shape and Motion from Image Streams using Non-Linear Least Squares, Digital Equipment Corporation, Cambridge Research Lab, Technical Report, March, 1993．
[4] Volker Blanz,Thomas Vetler. A Morphable Model for The Synthesis of 3D Facial. SIGGRAPH99, 1999．
[5] 吴思宁,具有真实感的特定人脸模型合成方法的研究,北京工业大学硕士毕业论文 1997
[6] F.I. Parke. Computer generated animation of faces. Proceedings ACM annual conference., August, 1972．
[7] Horace M.S.Ip and Lijun,Yin. Constructing a 3D Individualized Head Model from Two Orthogonal Views. The Visual Computer( 1996) 12:254-266．
[8] Frederic I.Parke and Keith Waters. Computer Facial Antimation, A.K Peters. Wellesley, Massachusetts, 1996．
[9] Takaaki Akimoto, Yasuhito Suenaga, and Richard S. Wallace, Automatic Creation of 3D Facial Models. IEEE Computer Graphics & Applications, Sep., 1993．
[10] Y.C. Lee, D. Terzopoulos, and Keith Waters. Constructing physics-based fa-cial models of individuals. Visual Computer, Proceedings of Graphics Interface 93:1-8, 1993
[11] Yuencheng Lee. Demetri Terzopoulos, and Keith Waters, Realistic Modeling for Facial Animation, In Computer Graphics (Proc. SIGGRAPH), pp. 55-62, 1996．
[12] Lee W. S., Kalra P., Magnenat-Thalmann N, "Model Based Face Reconstruction for Animation", In Proc. Multimedia Modeling (MMM'97) , Singapore, pp. 323-338,1997．
[13] D. DeCarlos, D. Metaxas, and M. Stone. An anthropometric face model us-ing variational techniques. In Computer Graphics Proceedings SIGGRAPH'98,
[14] B. Guenter, C. Grimm, D. Wolf, H. Malvar, and F. Pighin. Making faces. In Computer Graphics Proceedings SIGGRAPH'98, pages 55-66, 1998
[15] Z.liu, Z.Zhang,C.Jacobs and M.Cohen. Rapid Modeling of Animated Faces From Video. The Thirt International Conference on Visual Computing, Page: 58-67． Sep.2000．
[16] P. Fua and C. Miccio, Animated Heads from Ordinary Images: A Least Squares Approach, Computer Vision and Image Understanding, 1999, In Press.
[17] T. Cootes, C. Taylor, D. Cooper, J. Graham, Active Shape Models-Their Training and Application. Computer Vision and Image Understanding, January 1995, Vol. 61, No. 1, pp.38-59．
[18] T.F. Cootes, G.J. Edwards, and C.J. Taylor. Active appearance models. In Burkhardt and Neumann, editors. Computer Vision-ECCV'98 Vol. II, Freiburg, Germany, 1998． Springer, Lecture Notes in Computer Science 1407．
[19] A. Lanitis, C. J. Taylor, and T. F. Cootes. Automatic interpretation and coding of face images using flexible models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7) :743-756,1997．
[20] J. Miao, W.Gao, Y.Q.Chen, et al, Gravity-Center Template Based Human Face Feature Detection, Advances in Multimodal Interfaces-ICMI 2000, pp207-214．
[21] M. Kass, A. Witkin, and D. Terzopoulos. "Snakes: Active Contour Models", Internation Journal of Computer Vision, pp: 321-331, 1988．
[22] Williams, Donna and Shah, Mubarak. "A Fast Algorithm for Active Contours and Curvature Estimation", CVGIP: Image Understanding. Vol. 55, No. 1, January 1992． pp. 14-26．
[23] Stever.Gunn and Marks.Nixon, Global and local active contour for head boundary extraction, International Journal of Computer Vision, 30(1) , pp:43-54,1998．
[24] B. Li and N. Roeder, Face contour extraction from front-view images, Pattern recognition, 28(8) , 1167-1179, 1995．
[25] H.A. Rowley, S. Baluja, and T. Kanade, Neural network-based face detection, IEEE-PAMI, 20(1) , 23-38, 1998．
[26] Paul Debevec. A Neural Network for Facial Feature Location. UC Berkeley CS283 Project Report, December 1992． http://www.debevec.org/face recognition.html
[27] A.L. Yuille, Peter W.Hallinan and David S.Cohen, Feature extraction from faces using deformable templates, International Journal of Computer Vision, 8(2) : 99-111, 1992．
[28] J.Y. Deng and F. Lai, Region-based template deformation and masking for eye-feature
extraction and description, Pattern recognition, 30(3) , 403-419, 1997．
[29] S.H. Jeng, H.Y.M. Liao, et al, Facial feature detection using geometrical face model: an efficient approach, Pattern recognition, 31(3) , 273-282, 1998．
[30] K. Lam and H. Yan, Locating and extracting the eye in human face images, Pattern recognition, 29(5) . 771-779, 1996．
[31] G.Chow and X.Li, Towards a system for automatic facial feature detection, PR 26(12 ), 1739-1755, 1993．
[32] X.Jia and M.S.Nixon, Analysing front view face profile for face recognition via the Walsh transform. PRL 15, 551-558, 1994．
[33] X.Song. C.W.Lce and S.Tsuji, Extracting facial features with partial feature template, Proc. ACCV, 751-754, 1994．
[34] T.C.Chang, T.S.Huang and C.Novak, Facial feature extraction from color images, Proc. IJCPR, 39-43, 1994．
[35] X.Xie, R.Sudhakar and H.Zhuang, On improving eye feature extraction using deformable templates, Pattern recognition, 27(6) ,pp: 791-799,1994．
[36] Kin-Man.Lam and Hong Yan, Location and extracting the eye in human face images, Pattern recognition. 29(5) .771-779,1996．
[37] D. Jang and H.Choi, Active models for tracking moving objects, Pattern recognition, 33(2000) ,1135-1146,2000．
[38] T.F.Cootes,C.J.Taylor,D.H.Cooper And J.Graham, Active Shape Models-Their Training and Application. Computer Vision Graphics and Image Understanding, Vol61,No l.pp 38-59,1995．
[39] A.Ianitis.C.J.Taylor.T.F.Cootes and T.Ahmed. Automatic Interpretation of Human Faces and Hand Gestures Using Flexible Temolates. To appear in the proceedings of the International Workshop on Face and Gesture Recognition, 1995．
[40] Shimizu.l, Zhang.Z, Akamatsu.S etc. Head pose determination from one image using a generic model. 3rd IEEE International conference on automatic face and gesture recognition, pp.100-105, Nara, Japan, April. 1998．
[41] Volker Kr(?)ger, Sven Brims and Gerald Sommer Efficient Head Pose Estimation with Gabor Wavelet Networks BMVC, Bristol, Great Britain, 11-14 September 2000．