机器仿生眼的多任务学习人脸分析
|
摘要
智能机器人中人机交互的性能至关重要,人脸分析可以使人机交互变得更友善.文中提出可以同时进行笑容识别和性别分类的多任务学习卷积神经网络,同时学习存在内在相关性的任务,提升单个任务的性能.在Celeb A数据集的测试集上,文中网络在笑容识别任务和性别分类任务中均获取较高准确率.在设计的机器仿生眼上验证文中模型,获得良好的笑容识别效果和性别分类效果.文中对人脸分析进行的研究可以提升与机器仿生眼人机交互的能力.
The performance of human-machine interaction is crucial for intelligence robot,and face analysis makes human-machine interaction more friendly. In this paper, a multi-task learning convolutional neural network is proposed. The tasks of smile recognition and gender classification are solved simultaneously. Inherent correlated tasks are learned,and the performance of individual task is improved. On CelebA test dataset,the proposed network achieves high accuracy on a smile recognition task and a gender classification task. The proposed model is tested on the designed machine bionic vision eyes,achieving satisfactory result on smile recognition and gender classification. The research on face analysis in this paper improves the human-machine interaction ability with the machine bionic eyes.
The performance of human-machine interaction is crucial for intelligence robot,and face analysis makes human-machine interaction more friendly. In this paper, a multi-task learning convolutional neural network is proposed. The tasks of smile recognition and gender classification are solved simultaneously. Inherent correlated tasks are learned,and the performance of individual task is improved. On CelebA test dataset,the proposed network achieves high accuracy on a smile recognition task and a gender classification task. The proposed model is tested on the designed machine bionic vision eyes,achieving satisfactory result on smile recognition and gender classification. The research on face analysis in this paper improves the human-machine interaction ability with the machine bionic eyes.
引文
[1]TAIGMAN Y,YANG M,RANZATO M A,et al.Deepface:Closing the Gap to Human-Level Performance in Face Verification//Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2014:1701-1708.
[2]SUN Y,WANG X G,TANG X O.Deep Convolutional Network Cascade for Facial Point Detection//Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2013:3476-3483.
[3]CAI H B,LIU B L,ZHANG J H,et al.Visual Focus of Attention Estimation Using Eye Center Localization.IEEE Systems Journal,2017,11(3):1320-1325.
[4]LIU M Y,LI S X,SHAN S G,et al.AU-Aware Deep Networks for Facial Expression Recognition//Proc of the 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition.Washington,USA:IEEE,2013.DOI:10.1109/FG.2013.6553734.
[5]BALUJA S,ROWLEY H A.Boosting Sex Identification Performance.International Journal of Computer Vision,2007,71(1):111-119.
[6]GLAUNER P O.Deep Learning For Smile Recognition//Proc of the 12th Conference on Uncertainty Modelling in Knowledge Engineering and Decision Making.Berlin,Germany:Springer,2016:319-324.
[7]ANTIPOV G,BERRANI S A,DUGELARY J L.Minimalistic CNN-Based Ensemble Model for Gender Prediction from Face Images.Pattern Recognition Letters,2016,70:59-65.
[8]CHEN D,REN S Q,WEI Y C,et al.Joint Cascade Face Detection and Alignment//Proc of the European Conference on Computer Vision.Berlin,Germany:Springer,2014:109-122.
[9]CARUANA R.Multitask Learning.Machine Learning.1997,28(1):41-75.
[10]EIGEN D,FERGUS R.Predicting Depth,Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture[C/OL].[2018-08-28].https://arxiv.org/pdf/1411.4734.pdf.
[11]KOKKINOS I.UberNet:Training a Universal Convolutional Neural Network for Low-,Mid-,and High-Level Vision Using Diverse Datasets and Limited Memory[C/OL].[2018-08-28].https://arxiv.org/pdf/1609.02132.pdf.
[12]ZHU X X,RAMANAN D.Face Detection,Pose Estimation,and Landmark Localization in the Wild//Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2012:2879-2886.
[13]ZHANG K H,HUANG Y Z,WU H,et al.Facial Smile Detection Based on Deep Learning Features//Proc of the 3rd IAPR Asian Conference on Pattern Recognition.Washington,USA:IEEE,2015:534-538.
[14]MAVADATI S M,MAHOOR M H,BARTLETT K,et al.DISFA:A Spontaneous Facial Action Intensity Database.IEEE Transactions on Affective Computing,2013,4(2):151-160.
[15]CHEN J K,OU Q H,CHI Z R,et al.Smile Detection in the Wild with Deep Convolutional Neural Networks.Machine Vision and Applications,2017,28(1/2):173-183.
[16]NIAN F D,LI L Y,LI T,et al.Robust Gender Classification on Unconstrained Face Images//Proc of the 7th International Conference on Internet Multimedia Computing and Service.New York,USA:ACM,2015.DOI:10.1145/2808492.2808570.
[17]HUANG G B,MATTAR M,BERG T,et al.Labeled Faces in the Wild:A Database for Studying Face Recognition in Unconstrained Environments[C/OL].[2018-08-28].http://w.tamaraberg.com/papers/Huang_eccv2008-lfw.pdf.
[18]VAN DE WOLFSHAAR J,KARAABA M F,WIERING M A.Deep Convolutional Neural Networks and Support Vector Machines for Gender Recognition//Proc of the IEEE Symposium Series on Computing Intelligence.Washington,USA:IEEE,2015:188-195.
[19]PHILLIPS P J,WECHSLER H,HUANG J,et al.The FERETDatabase and Evaluation Procedure for Face-Recognition Algorithms.Image and Vision Computing,1998,16(5):295-306.
[20]PHILLIPS P J,MOON H,RIZVI S A,et al.The FERET Evaluation Methodology for Face-Recognition Algorithms.IEEE Transactions on Pattern Analysis and Machine Intelligence,2000,22(10):1090-1104.
[21]EIDINGER E,ENBAR R,HASSNER T.Age and Gender Estimation of Unfiltered Faces.IEEE Transactions on Information Forensics and Security,2014,9(12):2170-2179.
[22]MANSANET J,ALBIOL A,PAREDES R.Local Deep Neural Networks for Gender Recognition.Pattern Recognition Letters,2016,70:80-86.
[23]GALLAGHER A C,CHEN T.Understanding Images of Groups of People//Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2009:256-263.
[24]KUINDERSMA S,DEITS R,FALLON M,et al.OptimizationBased Locomotion Planning,Estimation,and Control Design for the Atlas Humanoid Robot.Autonomous Robots,2016,40(3):429-455.
[25]KANEKO K,HARADA K,KANEHIRO F,et al.Humanoid Robot HRP-3//Proc of the IEEE/RSJ International Conference on Intelligent Robots and Systems.Washington,USA:IEEE,2008:2471-2478.
[26]SAKAGAMI Y,WATANABE R,AOYAMA C,et al.The Intelligent ASIMO:System Overview and Integration//Proc of the IEEE/RSJ International Conference on Intelligent Robots and Systems.Washington,USA:IEEE,2002,III:2478-2483.
[27]WIKIPEDIA.Head and Neck Anatomy[J/OL].[2018-08-28].http://en.wikipedia.org/wiki/Head_and_neck_anatomy.
[28]ZATSIORSKY V M.Kinematics of Human Motion.Champaign,USA:Human Kinetics,1998.
[29]WIKIPEDIA.Human Eye[J/OL].[2018-08-28].http://en.wikipedia.org/wiki/Human_eye.
[30]ZEILER M D,FERGUS R.Visualizing and Understanding Convolutional Networks//Proc of the European Conference on Computer Vision.Berlin,Germany:Springer,2014:818-833.
[31]HE K M,ZHANG X Y,REN S Q,et al.Deep Residual Learning for Image Recognition//Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2016:770-778.
[32]SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition[C/OL].[2018-08-28].https://arxiv.org/pdf/1409.1556.pdf.
[33]LIU Z W,LUO P,WANG X G,et al.Deep Learning Face Attributes in the Wild//Proc of the IEEE International Conference on Computer Vision.Washington,USA:IEEE,2015:3730-3738.
[34]ZHANG N,PALURI M,RANZATO M A,et al.Panda:Pose Aligned Networks for Deep Attribute Modeling//Proc of the IEEEConference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2014:1637-1644.
[35]EHRLICH M,SHIELDS T J,ALMAEV T,et al.Facial Attributes Classification Using Multi-task Representation Learning//Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2016:47-55.
[36]RANJAN R,PATEL V M,CHELLAPPA R.Hyperface:A Deep Multi-task Learning Framework for Face Detection,Landmark Localization,Pose Estimation,and Gender Recognition[J/OL].[2018-08-28].https://arxiv.org/pdf/1603.01249.pdf.
[2]SUN Y,WANG X G,TANG X O.Deep Convolutional Network Cascade for Facial Point Detection//Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2013:3476-3483.
[3]CAI H B,LIU B L,ZHANG J H,et al.Visual Focus of Attention Estimation Using Eye Center Localization.IEEE Systems Journal,2017,11(3):1320-1325.
[4]LIU M Y,LI S X,SHAN S G,et al.AU-Aware Deep Networks for Facial Expression Recognition//Proc of the 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition.Washington,USA:IEEE,2013.DOI:10.1109/FG.2013.6553734.
[5]BALUJA S,ROWLEY H A.Boosting Sex Identification Performance.International Journal of Computer Vision,2007,71(1):111-119.
[6]GLAUNER P O.Deep Learning For Smile Recognition//Proc of the 12th Conference on Uncertainty Modelling in Knowledge Engineering and Decision Making.Berlin,Germany:Springer,2016:319-324.
[7]ANTIPOV G,BERRANI S A,DUGELARY J L.Minimalistic CNN-Based Ensemble Model for Gender Prediction from Face Images.Pattern Recognition Letters,2016,70:59-65.
[8]CHEN D,REN S Q,WEI Y C,et al.Joint Cascade Face Detection and Alignment//Proc of the European Conference on Computer Vision.Berlin,Germany:Springer,2014:109-122.
[9]CARUANA R.Multitask Learning.Machine Learning.1997,28(1):41-75.
[10]EIGEN D,FERGUS R.Predicting Depth,Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture[C/OL].[2018-08-28].https://arxiv.org/pdf/1411.4734.pdf.
[11]KOKKINOS I.UberNet:Training a Universal Convolutional Neural Network for Low-,Mid-,and High-Level Vision Using Diverse Datasets and Limited Memory[C/OL].[2018-08-28].https://arxiv.org/pdf/1609.02132.pdf.
[12]ZHU X X,RAMANAN D.Face Detection,Pose Estimation,and Landmark Localization in the Wild//Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2012:2879-2886.
[13]ZHANG K H,HUANG Y Z,WU H,et al.Facial Smile Detection Based on Deep Learning Features//Proc of the 3rd IAPR Asian Conference on Pattern Recognition.Washington,USA:IEEE,2015:534-538.
[14]MAVADATI S M,MAHOOR M H,BARTLETT K,et al.DISFA:A Spontaneous Facial Action Intensity Database.IEEE Transactions on Affective Computing,2013,4(2):151-160.
[15]CHEN J K,OU Q H,CHI Z R,et al.Smile Detection in the Wild with Deep Convolutional Neural Networks.Machine Vision and Applications,2017,28(1/2):173-183.
[16]NIAN F D,LI L Y,LI T,et al.Robust Gender Classification on Unconstrained Face Images//Proc of the 7th International Conference on Internet Multimedia Computing and Service.New York,USA:ACM,2015.DOI:10.1145/2808492.2808570.
[17]HUANG G B,MATTAR M,BERG T,et al.Labeled Faces in the Wild:A Database for Studying Face Recognition in Unconstrained Environments[C/OL].[2018-08-28].http://w.tamaraberg.com/papers/Huang_eccv2008-lfw.pdf.
[18]VAN DE WOLFSHAAR J,KARAABA M F,WIERING M A.Deep Convolutional Neural Networks and Support Vector Machines for Gender Recognition//Proc of the IEEE Symposium Series on Computing Intelligence.Washington,USA:IEEE,2015:188-195.
[19]PHILLIPS P J,WECHSLER H,HUANG J,et al.The FERETDatabase and Evaluation Procedure for Face-Recognition Algorithms.Image and Vision Computing,1998,16(5):295-306.
[20]PHILLIPS P J,MOON H,RIZVI S A,et al.The FERET Evaluation Methodology for Face-Recognition Algorithms.IEEE Transactions on Pattern Analysis and Machine Intelligence,2000,22(10):1090-1104.
[21]EIDINGER E,ENBAR R,HASSNER T.Age and Gender Estimation of Unfiltered Faces.IEEE Transactions on Information Forensics and Security,2014,9(12):2170-2179.
[22]MANSANET J,ALBIOL A,PAREDES R.Local Deep Neural Networks for Gender Recognition.Pattern Recognition Letters,2016,70:80-86.
[23]GALLAGHER A C,CHEN T.Understanding Images of Groups of People//Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2009:256-263.
[24]KUINDERSMA S,DEITS R,FALLON M,et al.OptimizationBased Locomotion Planning,Estimation,and Control Design for the Atlas Humanoid Robot.Autonomous Robots,2016,40(3):429-455.
[25]KANEKO K,HARADA K,KANEHIRO F,et al.Humanoid Robot HRP-3//Proc of the IEEE/RSJ International Conference on Intelligent Robots and Systems.Washington,USA:IEEE,2008:2471-2478.
[26]SAKAGAMI Y,WATANABE R,AOYAMA C,et al.The Intelligent ASIMO:System Overview and Integration//Proc of the IEEE/RSJ International Conference on Intelligent Robots and Systems.Washington,USA:IEEE,2002,III:2478-2483.
[27]WIKIPEDIA.Head and Neck Anatomy[J/OL].[2018-08-28].http://en.wikipedia.org/wiki/Head_and_neck_anatomy.
[28]ZATSIORSKY V M.Kinematics of Human Motion.Champaign,USA:Human Kinetics,1998.
[29]WIKIPEDIA.Human Eye[J/OL].[2018-08-28].http://en.wikipedia.org/wiki/Human_eye.
[30]ZEILER M D,FERGUS R.Visualizing and Understanding Convolutional Networks//Proc of the European Conference on Computer Vision.Berlin,Germany:Springer,2014:818-833.
[31]HE K M,ZHANG X Y,REN S Q,et al.Deep Residual Learning for Image Recognition//Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2016:770-778.
[32]SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition[C/OL].[2018-08-28].https://arxiv.org/pdf/1409.1556.pdf.
[33]LIU Z W,LUO P,WANG X G,et al.Deep Learning Face Attributes in the Wild//Proc of the IEEE International Conference on Computer Vision.Washington,USA:IEEE,2015:3730-3738.
[34]ZHANG N,PALURI M,RANZATO M A,et al.Panda:Pose Aligned Networks for Deep Attribute Modeling//Proc of the IEEEConference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2014:1637-1644.
[35]EHRLICH M,SHIELDS T J,ALMAEV T,et al.Facial Attributes Classification Using Multi-task Representation Learning//Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2016:47-55.
[36]RANJAN R,PATEL V M,CHELLAPPA R.Hyperface:A Deep Multi-task Learning Framework for Face Detection,Landmark Localization,Pose Estimation,and Gender Recognition[J/OL].[2018-08-28].https://arxiv.org/pdf/1603.01249.pdf.