基于深度学习的人脸表情迁移方法
|
摘要
针对人脸表情迁移生成图像质量不高、训练过程较长且生成速度较慢的问题,文中提出了一种基于生成式对抗网络的人脸表情迁移方法,使表情迁移更加快速和自然。首先,利用卷积神经网络进行人脸特征提取,并将图像从高维空间映射到浅层空间,在浅层空间中利用生成式对抗网络模型对人脸表情特征进行判别;然后,通过最近邻上采样层和卷积层组合结构将图像从浅层空间映射到高维空间,并在此过程中通过加入表情标签特征图对人脸表情进行改变。与Fader Networks相比,所提方法的网络模型参数量减少43.7%,训练时间缩短了36%。实验结果表明,所提方法有效地提高了人脸表情迁移生成图像的速度和质量。
In order to solve the problems of low image quality,long training process and slow generation speed of face expression transfer,this paper proposed a facial expression transfermethod based on generative adversarial network to make expression transfer faster and more natural.Firstly,the facial features are extracted by using convolutional neural network,and the images are mapped from high-dimensional space to shallow space.In the shallow space,the facial expression features are discriminated by using the Generative Adversarial Networks.Then the nearest neighbors up-sampling and convolutional neural networks are used to mapthe image from the shallow space to the high-dimensional space,and in this process,the face expression is changed by adding the facial expression feature maps into neural networks.Compared with Fader Networks,the network model parameter amount of the proposed method is reduced by 43.7% and training time is reduced by 36%.The experimental results show that the proposed method can effectively improve the quality and the speed of generated images.
In order to solve the problems of low image quality,long training process and slow generation speed of face expression transfer,this paper proposed a facial expression transfermethod based on generative adversarial network to make expression transfer faster and more natural.Firstly,the facial features are extracted by using convolutional neural network,and the images are mapped from high-dimensional space to shallow space.In the shallow space,the facial expression features are discriminated by using the Generative Adversarial Networks.Then the nearest neighbors up-sampling and convolutional neural networks are used to mapthe image from the shallow space to the high-dimensional space,and in this process,the face expression is changed by adding the facial expression feature maps into neural networks.Compared with Fader Networks,the network model parameter amount of the proposed method is reduced by 43.7% and training time is reduced by 36%.The experimental results show that the proposed method can effectively improve the quality and the speed of generated images.
引文
[1] IOFFE S,SZEGEDY C:Batch Normalization:Accelerating Deep Network Training by Reducing Internal Covariate Shift[J].ar-Xiv:1502.03167v3,2015.
[2] ISOLA P,ZHU J Y,ZHOU T H,et al.Image-to-image translation with conditional adversarial networks[J].arXiv:1611.07004,2016.
[3] 王娅,侯进,王献.基于顶点权重的网格简化在虚拟人脸中的应用[J].计算机仿真,2014,31(2):329-334.
[4] 雷腾.虚拟人眼的运动与表情合成的研究[D].成都:西南交通大学,2014.
[5] 李俊龙,章登义,黄珺.Kinect驱动的人脸动画合成技术研究[J].计算机工程,2015,41(3):237-241.
[6] ZHANG H,XU T,LI H,et al.Stackgan:Text to photo-realistic image synthesis with stacked generative adversarial networks[J].arXiv:1612.03242,2016.
[7] ZHANG Z,SONG Y,QI H.Age progression/regression by conditional adversarial autoencoder[C]//The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2017.
[8] ZHAO J,MATHIEU M,LECUN Y.Energy-based generative adversarial network[C]//5th International Conference on Learning Representations (ICLR).2017.
[9] LAMPLE G,ZEGHIDOUR N.Fader Networks:Manipulating Images by Sliding Attributes[J].arXiv:1706.00409v2,2017.
[10] HINTON G,KRIZHEVSKY A,WANG S.Transforming auto-encoders[C]//Artificial Neural Networks and Machine Learning(ICANN 2011).2011:44-51.
[11] PERARNAU G,VAN DE WEIJER J,RADUCANU B,et al.Invertible conditional gans for image editing[J].arXiv:1611.06355,2016.
[12] RATLIFF L J,BURDEN S A,SASTRY S S.Characterization and computation of local Nash equilibria in continuous games[C]//Proceedings of the 51st Annual Allerton Conference on Communication,Control,and Computing (Allerton).Monticello,IL,USA,IEEE,2013:917-924.
[13] ARJOVSKY M,OTTOU L.Towards principled methods for training generative adversarial networks[C]//ICLR.2017.
[14] SHEN W,LIU R.Learning residual images for face attribute manipulation[C]//The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).2017.
[15] ANTIPOV G,BACCOUCHE M,DUGELAY J L.Face aging with conditional generative adversarial networks[J].arXiv:1702.01983,2017.
[16] LIU Z W,LUO P,WANG X G,et al.Deep learning face attri- butes in the wild[C]//Proceedings of International Conference on Computer Vision (ICCV).2015.
[17] GLOROT X,BENGIO Y.Understanding the difficulty of training deep feedforward neural networks[C]//AISTATS.2010.
[18] KINGMA D,BA J.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2014.
[2] ISOLA P,ZHU J Y,ZHOU T H,et al.Image-to-image translation with conditional adversarial networks[J].arXiv:1611.07004,2016.
[3] 王娅,侯进,王献.基于顶点权重的网格简化在虚拟人脸中的应用[J].计算机仿真,2014,31(2):329-334.
[4] 雷腾.虚拟人眼的运动与表情合成的研究[D].成都:西南交通大学,2014.
[5] 李俊龙,章登义,黄珺.Kinect驱动的人脸动画合成技术研究[J].计算机工程,2015,41(3):237-241.
[6] ZHANG H,XU T,LI H,et al.Stackgan:Text to photo-realistic image synthesis with stacked generative adversarial networks[J].arXiv:1612.03242,2016.
[7] ZHANG Z,SONG Y,QI H.Age progression/regression by conditional adversarial autoencoder[C]//The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2017.
[8] ZHAO J,MATHIEU M,LECUN Y.Energy-based generative adversarial network[C]//5th International Conference on Learning Representations (ICLR).2017.
[9] LAMPLE G,ZEGHIDOUR N.Fader Networks:Manipulating Images by Sliding Attributes[J].arXiv:1706.00409v2,2017.
[10] HINTON G,KRIZHEVSKY A,WANG S.Transforming auto-encoders[C]//Artificial Neural Networks and Machine Learning(ICANN 2011).2011:44-51.
[11] PERARNAU G,VAN DE WEIJER J,RADUCANU B,et al.Invertible conditional gans for image editing[J].arXiv:1611.06355,2016.
[12] RATLIFF L J,BURDEN S A,SASTRY S S.Characterization and computation of local Nash equilibria in continuous games[C]//Proceedings of the 51st Annual Allerton Conference on Communication,Control,and Computing (Allerton).Monticello,IL,USA,IEEE,2013:917-924.
[13] ARJOVSKY M,OTTOU L.Towards principled methods for training generative adversarial networks[C]//ICLR.2017.
[14] SHEN W,LIU R.Learning residual images for face attribute manipulation[C]//The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).2017.
[15] ANTIPOV G,BACCOUCHE M,DUGELAY J L.Face aging with conditional generative adversarial networks[J].arXiv:1702.01983,2017.
[16] LIU Z W,LUO P,WANG X G,et al.Deep learning face attri- butes in the wild[C]//Proceedings of International Conference on Computer Vision (ICCV).2015.
[17] GLOROT X,BENGIO Y.Understanding the difficulty of training deep feedforward neural networks[C]//AISTATS.2010.
[18] KINGMA D,BA J.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2014.