基于困难样本三元组损失的多任务行人再识别
|
摘要
针对现有行人再识别算法中采用单一外观特征所存在的特征判别力的不足问题,在深度学习框架下提出一种基于困难样本三元组损失的多任务行人再识别网络,以同时学习身份和属性标签的方式获得更多的行人判别信息.首先将预处理后的图片输入预训练的ResNet-50模型中提取行人特征信息;然后将其特征输入所设计的多任务网络,通过最小化身份和属性联合三元组损失调整网络模型参数,共同完成行人身份和属性判别双分支网络的训练;最后用训练好的模型提取行人特征,用于行人再识别任务,同时实现行人属性的判断.在Market-1501和DukeMTMC-reID这2个数据集上的实验证明,文中网络在行人再识别任务中所提取的特征更具有表征力,识别精度优于现有方法,并且能完成属性识别任务.
Aiming to enhance features discrimination for person re-identification, a deep multi-task person re-identification network based on TriHard loss is proposed. By learning identity and attributes labels simultaneously, the network can achieve more discriminative information of pedestrians. Firstly, the pre-trained ResNet-50 is loaded to extract pedestrian features of pre-processed images. Secondly, pedestrian features are fed into the designed multi-task network which consists of two branches. The two branches are trained jointly by minimizing combined TriHard loss of identity and attribute. Finally, the trained model is used to extract pedestrian appearances and attributes features. The features are used for person re-identification and attributes recognition. From the experimental results on the Market-1501 dataset and the DukeMTMC-reID dataset, it shows that features extracted from the proposed network are more discriminative. The multitask network achieves higher identification accuracy over the state-of-the-arts, together with the person attributes recognition.
Aiming to enhance features discrimination for person re-identification, a deep multi-task person re-identification network based on TriHard loss is proposed. By learning identity and attributes labels simultaneously, the network can achieve more discriminative information of pedestrians. Firstly, the pre-trained ResNet-50 is loaded to extract pedestrian features of pre-processed images. Secondly, pedestrian features are fed into the designed multi-task network which consists of two branches. The two branches are trained jointly by minimizing combined TriHard loss of identity and attribute. Finally, the trained model is used to extract pedestrian appearances and attributes features. The features are used for person re-identification and attributes recognition. From the experimental results on the Market-1501 dataset and the DukeMTMC-reID dataset, it shows that features extracted from the proposed network are more discriminative. The multitask network achieves higher identification accuracy over the state-of-the-arts, together with the person attributes recognition.
引文
[1]Zheng Z D,Zheng L,Yang Y.A discriminatively learned CNNembedding for person re-identification[J].ACM Transactions on Multimedia Computing Communications and Applications,2018,14(1):Article No.13
[2]Varior R R,Haloi M,Wang G.Gated Siamese convolutional neural network architecture for human re-identification[C]//Proceedings of the European Conference on Computer Vision.Heidelberg:Springer,2016:791-808
[3]Cheng D,Gong Y H,Zhou S P,et al.Person re-identification by multi-channel parts-based CNN with improved triplet loss function[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2016:1335-1344
[4]Chen W H,Chen X T,Zhang J G,et al.Beyond triplet loss:a deep quadruplet network for person re-identification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:403-412
[5]Hermans A,Beyer L,Leibe B.In defense of the triplet loss for person re-identification[OL].[2018-08-05].https://arxiv.org/abs/1703.07737
[6]Xiao Q Q,Luo H,Zhang C.Margin sample mining loss:a deep learning based method for person re-identification[OL].[2018-08-05].https://arxiv.org/abs/1710.00478
[7]An L,Chen X J,Kafai M,et al.Improving person re-identification by soft biometrics based re-ranking[C]//Proceedings of the 7th International Conference on Distributed Smart Cameras.Los Alamitos:IEEE Computer Society Press,2013:1-6
[8]Layne R,Hospedales T,Gong S G.Re-id:hunting attributes in the wild[C]//Proceedings of British Machine Vision Conference.Guildford:BMVA Press,2014:Article No.1
[9]Layne R,Hospedales T,Gong S G.Person re-identification by attributes[C]//Proceedings of British Machine Vision Conference.Guildford:BMVA Press,2012:Article No.24
[10]Su C,Yang F,Zhang S L,et al.Multi-task learning with low rank attribute embedding for person re-identification[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2015:3739-3747
[11]Khamis S,Kuo C H,Singh V K,et al.Joint learning for attribute-consistent person re-identification[C]//Proceedings of European Conference on Computer Vision.Heidelberg:Springer,2014:134-146
[12]Zhu J Q,Liao S C,Lei Z,et al.Multi-label convolutional neural network based pedestrian attribute classification[J].Image and Vision Computing,2017,58:224-229
[13]Matsukawa T,Suzuki E.Person re-identification using CNNfeatures learned from combination of attributes[C]//Proceedings of the 23th International Conference on Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:2428-2433
[14]Schumann A,Stiefelhagen R.Person re-identification by deep learning attribute-complementary information[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:20-28
[15]Lin Y T,Zheng L,Zheng Z D,et al.Improving person re-identification by attribute and identity learning[OL].[2018-08-05].https://arxiv.org/abs/1703.07220
[16]Deng Y B,Luo P,Loy C C,et al.Pedestrian attribute recognition at far distance[C]//Proceedings of the 22nd ACM International Conference on Multimedia.New York:ACM Press,2014:789-792
[17]Li D W,Zhang Z,Chen X T,et al.A richly annotated dataset for pedestrian attribute recognition[OL].[2018-08-05].https://arxiv.org/abs/1603.07054
[18]Sun Y F,Zheng L,Yang Y,et al.Beyond part models:person retrieval with refined part pooling[C]//Proceedings of the EuEuropean Conference on Computer Vision.Heidelberg:Springer,2018:480-496
[19]Zheng L,Shen L Y,Tian L,et al.Scalable person re-identification:a benchmark[C]//Proceedings of the IEEEInternational Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2015:1116-1124
[20]Ristani E,Solera F,Zou R,et al.Performance measures and a data set for multi-target,multi-camera tracking[C]//Proceedings of the European Conference on Computer Vision.Heidelberg:Springer,2016:17-35
[21]Zhong Z,Zheng L,Cao D L,et al.Re-ranking person re-identification with k-reciprocal encoding[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:1318-1327
[22]Ristani E,Tomasi C.Features for multi-target multi-camera tracking and re-Identification[C]//Proceedings of the IEEEConference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2018:6036-6046
[23]Ustinova E,Ganin Y,Lempitsky V.Multi-region bilinear convolutional neural networks for person re-identification[C]//Proceedings of the 14th IEEE International Conference on Advanced Video and Signal Based Surveillance.Los Alamitos:IEEE Computer Society Press,2017:1-6
[24]Zhou S P,Wang J J,Wang J Y,et al.Point to set similarity based deep feature learning for person re-identification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:3741-3750
[25]Lin J,Ren L L,Lu J W,et al.Consistent-aware deep learning for person re-identification in a camera network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:5771-5780
[26]Zheng Z D,Zheng L,Yang Y.Unlabeled samples generated by GAN improve the person re-identification baseline in vitro[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2017:3754-3762
[27]Sun Y F,Zheng L,Deng W J,et al.SVDNet for pedestrian retrieval[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2017:3800-3808
[28]Wu Yancheng,Chen Hongchang,Li Shaomei,et al.Person re-identification using attribute priori distribution[OL].[2018-08-05].https://doi.org/10.16383/j.aas.c170691(in Chinese)(吴彦丞,陈鸿昶,李邵梅,等.基于行人属性先验分布的行人再识别[OL].[2018-08-05].https://doi.org/10.16383/j.aas.c170691)
[29]Saquib Sarfraz M,Schumann A,Eberle A,et al.Apose-sensitive embedding for person re-identification with expanded cross neighborhood re-ranking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2018:420-429
[2]Varior R R,Haloi M,Wang G.Gated Siamese convolutional neural network architecture for human re-identification[C]//Proceedings of the European Conference on Computer Vision.Heidelberg:Springer,2016:791-808
[3]Cheng D,Gong Y H,Zhou S P,et al.Person re-identification by multi-channel parts-based CNN with improved triplet loss function[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2016:1335-1344
[4]Chen W H,Chen X T,Zhang J G,et al.Beyond triplet loss:a deep quadruplet network for person re-identification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:403-412
[5]Hermans A,Beyer L,Leibe B.In defense of the triplet loss for person re-identification[OL].[2018-08-05].https://arxiv.org/abs/1703.07737
[6]Xiao Q Q,Luo H,Zhang C.Margin sample mining loss:a deep learning based method for person re-identification[OL].[2018-08-05].https://arxiv.org/abs/1710.00478
[7]An L,Chen X J,Kafai M,et al.Improving person re-identification by soft biometrics based re-ranking[C]//Proceedings of the 7th International Conference on Distributed Smart Cameras.Los Alamitos:IEEE Computer Society Press,2013:1-6
[8]Layne R,Hospedales T,Gong S G.Re-id:hunting attributes in the wild[C]//Proceedings of British Machine Vision Conference.Guildford:BMVA Press,2014:Article No.1
[9]Layne R,Hospedales T,Gong S G.Person re-identification by attributes[C]//Proceedings of British Machine Vision Conference.Guildford:BMVA Press,2012:Article No.24
[10]Su C,Yang F,Zhang S L,et al.Multi-task learning with low rank attribute embedding for person re-identification[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2015:3739-3747
[11]Khamis S,Kuo C H,Singh V K,et al.Joint learning for attribute-consistent person re-identification[C]//Proceedings of European Conference on Computer Vision.Heidelberg:Springer,2014:134-146
[12]Zhu J Q,Liao S C,Lei Z,et al.Multi-label convolutional neural network based pedestrian attribute classification[J].Image and Vision Computing,2017,58:224-229
[13]Matsukawa T,Suzuki E.Person re-identification using CNNfeatures learned from combination of attributes[C]//Proceedings of the 23th International Conference on Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:2428-2433
[14]Schumann A,Stiefelhagen R.Person re-identification by deep learning attribute-complementary information[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:20-28
[15]Lin Y T,Zheng L,Zheng Z D,et al.Improving person re-identification by attribute and identity learning[OL].[2018-08-05].https://arxiv.org/abs/1703.07220
[16]Deng Y B,Luo P,Loy C C,et al.Pedestrian attribute recognition at far distance[C]//Proceedings of the 22nd ACM International Conference on Multimedia.New York:ACM Press,2014:789-792
[17]Li D W,Zhang Z,Chen X T,et al.A richly annotated dataset for pedestrian attribute recognition[OL].[2018-08-05].https://arxiv.org/abs/1603.07054
[18]Sun Y F,Zheng L,Yang Y,et al.Beyond part models:person retrieval with refined part pooling[C]//Proceedings of the EuEuropean Conference on Computer Vision.Heidelberg:Springer,2018:480-496
[19]Zheng L,Shen L Y,Tian L,et al.Scalable person re-identification:a benchmark[C]//Proceedings of the IEEEInternational Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2015:1116-1124
[20]Ristani E,Solera F,Zou R,et al.Performance measures and a data set for multi-target,multi-camera tracking[C]//Proceedings of the European Conference on Computer Vision.Heidelberg:Springer,2016:17-35
[21]Zhong Z,Zheng L,Cao D L,et al.Re-ranking person re-identification with k-reciprocal encoding[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:1318-1327
[22]Ristani E,Tomasi C.Features for multi-target multi-camera tracking and re-Identification[C]//Proceedings of the IEEEConference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2018:6036-6046
[23]Ustinova E,Ganin Y,Lempitsky V.Multi-region bilinear convolutional neural networks for person re-identification[C]//Proceedings of the 14th IEEE International Conference on Advanced Video and Signal Based Surveillance.Los Alamitos:IEEE Computer Society Press,2017:1-6
[24]Zhou S P,Wang J J,Wang J Y,et al.Point to set similarity based deep feature learning for person re-identification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:3741-3750
[25]Lin J,Ren L L,Lu J W,et al.Consistent-aware deep learning for person re-identification in a camera network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:5771-5780
[26]Zheng Z D,Zheng L,Yang Y.Unlabeled samples generated by GAN improve the person re-identification baseline in vitro[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2017:3754-3762
[27]Sun Y F,Zheng L,Deng W J,et al.SVDNet for pedestrian retrieval[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2017:3800-3808
[28]Wu Yancheng,Chen Hongchang,Li Shaomei,et al.Person re-identification using attribute priori distribution[OL].[2018-08-05].https://doi.org/10.16383/j.aas.c170691(in Chinese)(吴彦丞,陈鸿昶,李邵梅,等.基于行人属性先验分布的行人再识别[OL].[2018-08-05].https://doi.org/10.16383/j.aas.c170691)
[29]Saquib Sarfraz M,Schumann A,Eberle A,et al.Apose-sensitive embedding for person re-identification with expanded cross neighborhood re-ranking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2018:420-429