Faster R-CNN行人检测与再识别为一体的行人检索算法
|
摘要
为了缩小目前行人再识别算法与真实世界中行人检索任务之间在应用上的差距,将行人检测与再识别这2个模块融为一体,提出一种基于改进的FasterR-CNN的行人检索算法.首先采用对边框进行迭代回归的方法改进原FasterR-CNN中的候选行人边框精度;然后利用包含欧氏距离和余弦距离的混合相似性距离函数来增强网络对于行人相似度的辨识能力;最后利用中心损失函数对网络的损失函数进行改进,通过提高不同行人特征的可区分度,实现更加精准的目标行人检索功能.基于CUHK-SYSU数据集的仿真实验结果表明,该算法的累积匹配特性(CMC top-1)、平均精度均值(mAP)分别为81.6%和78.9%;与相关行人检索算法相比, CMC top-1提升3.0%～18.0%, mAP提升3.0%～23.0%.
For closing the gap between research of pedestrian re-identification and pedestrian search in real-world applications, this paper proposes a new pedestrian search method by fusing the pedestrian detection and re-identification modules based on the modified Faster R-CNN. Firstly, it used an iterative bounding box regression network to promote the precision of bounding boxes. Then to enhance similarity learning ability, it used a modified metric learning method named MSLF which consists both cosine distance and Euclidean distance. Finally it added center loss to the whole loss function of network. Center loss boosts the network's ability by extracting discriminative features of different pedestrians, and enables the network to achieve a better result for query pedestrian search. It performed simulation on a large scale benchmark dataset named CUHK-SYSU, the experimental results show that proposed method achieves 81.6% in CMC top-1, and 78.9% in mAP, which outperforms other paralleling methods about 3.0%-18.0% in CMC top-1 and 3.0%-23.0% in mAP.
For closing the gap between research of pedestrian re-identification and pedestrian search in real-world applications, this paper proposes a new pedestrian search method by fusing the pedestrian detection and re-identification modules based on the modified Faster R-CNN. Firstly, it used an iterative bounding box regression network to promote the precision of bounding boxes. Then to enhance similarity learning ability, it used a modified metric learning method named MSLF which consists both cosine distance and Euclidean distance. Finally it added center loss to the whole loss function of network. Center loss boosts the network's ability by extracting discriminative features of different pedestrians, and enables the network to achieve a better result for query pedestrian search. It performed simulation on a large scale benchmark dataset named CUHK-SYSU, the experimental results show that proposed method achieves 81.6% in CMC top-1, and 78.9% in mAP, which outperforms other paralleling methods about 3.0%-18.0% in CMC top-1 and 3.0%-23.0% in mAP.
引文
[1]Girshick R,Donahue J,Darrell T,et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2014:580-587
[2]He K M,Zhang X Y,Ren S Q,et al.Spatial pyramid pooling in deep convolutional networks for visual recognition[C]//Proceedings of the 13th European Conference on Computer Vision.Heidelberg:Springer,2014:346-361
[3]Girshick R.Fast R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEEComputer Society Press,2015:1440-1448
[4]Ren S Q,He K M,Girshick R,et al.Faster R-CNN:towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(6):1137-1149
[5]Wang Bin,Liu Yang,Tang Sheng,et al.Pedestrian detection with fusion of multi-models and intra-frame information[J].Journal of Computer-Aided Design&Computer Graphics,2017,29(3):444-449(in Chinese)(王斌,刘洋,唐胜,等.融合多模型和帧间信息的行人检测算法[J].计算机辅助设计与图形学学报,2017,29(3):444-449)
[6]Zhang L L,Lin L,Liang X D,et al.Is faster R-CNN doing well for pedestrian detection?[C]//Proceedings of the 14th European Conference on Computer Vision.Heidelberg:Springer,2016:443-457
[7]Ma B P,Su Y,Jurie F.Local descriptors encoded by Fisher vectors for person re-identification[C]//Proceedings of the 12th European Conference on Computer Vision.Heidelberg:Springer,2012:413-422
[8]Farenzena M,Bazzani L,Perina A,et al.Person re-identification by symmetry-driven accumulation of local features[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2010:2360-2367
[9]Hu Y,Liao S C,Lei Z,et al.Exploring structural information and fusing multiple features for person re-identification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2013:794-799
[10]Liao S C,Hu Y,Zhu X Y,et al.Person re-identification by local maximal occurrence representation and metric learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE,2015:2197-2206
[11]Peng Zhiyong,Chang Faliang,Liu Hongbin,et al.Person re-identification algorithm based on HSV model and keypoints matching[J].Journal of Optoelectronics Laser,2015,26(8):1575-1582(in Chinese)(彭志勇,常发亮,刘洪彬,等.基于HSV模型和特征点匹配的行人重识别算法[J].光电子?激光,2015,26(8):1575-1582)
[12]Weinberger K Q,Saul L K.Distance metric learning for large margin nearest neighbor classification[J].Journal of Machine Learning Research,2006,10:207-244
[13]Shi H L,Yang Y,Zhu X Y,et al.Embedding deep metric for person re-identification:a study against large variations[C]//Proceedings of the 14th European Conference on Computer Vision.Heidelberg:Springer,2016:732-748
[14]Xiao T,Li H S,Ouyang W L,et al.Learning deep feature representations with domain guided dropout for person re-identification[C]//Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:1249-1258
[15]Xu Y L,Ma B P,Huang R,et al.Person search in a scene by jointly modeling people commonness and person uniqueness[C]//Proceedings of the 22nd ACM International Conference on Multimedia.New York:ACM Press,2014:937-940
[16]Zheng L,Zhang H H,Sun S Y,et al.Person re-identification in the wild[C]//Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2017:3346-3355
[17]Xiao T,Li S,Wang B C,et al.End-to-end deep learning for person search[OL].[2017-12-31].http://www.ee.cuhk.edu.hk/~xgwang/PS/paper.pdf
[18]Xiao T,Li S,Wang B C,et al.Joint detection and identification feature learning for person search[C]//Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:3376-3385
[19]Wen Y D,Zhang K P,Li Z F,et al.A discriminative feature learning approach for deep face recognition[C]//Proceedings of the 14th European Conference on Computer Vision.Heidelberg:Springer,2016:499-515
[20]He K M,Zhang X Y,Ren S Q,et al.Deep residual learning for image recognition[C]//Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:770-778
[21]Jia Y Q,Shelhamer E,Donahue J,et al.Caffe:convolutional architecture for fast feature embedding[C]//Proceedings of the22nd ACM International Conference on Multimedia.New York:ACM Press,2014:675-678
[2]He K M,Zhang X Y,Ren S Q,et al.Spatial pyramid pooling in deep convolutional networks for visual recognition[C]//Proceedings of the 13th European Conference on Computer Vision.Heidelberg:Springer,2014:346-361
[3]Girshick R.Fast R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEEComputer Society Press,2015:1440-1448
[4]Ren S Q,He K M,Girshick R,et al.Faster R-CNN:towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(6):1137-1149
[5]Wang Bin,Liu Yang,Tang Sheng,et al.Pedestrian detection with fusion of multi-models and intra-frame information[J].Journal of Computer-Aided Design&Computer Graphics,2017,29(3):444-449(in Chinese)(王斌,刘洋,唐胜,等.融合多模型和帧间信息的行人检测算法[J].计算机辅助设计与图形学学报,2017,29(3):444-449)
[6]Zhang L L,Lin L,Liang X D,et al.Is faster R-CNN doing well for pedestrian detection?[C]//Proceedings of the 14th European Conference on Computer Vision.Heidelberg:Springer,2016:443-457
[7]Ma B P,Su Y,Jurie F.Local descriptors encoded by Fisher vectors for person re-identification[C]//Proceedings of the 12th European Conference on Computer Vision.Heidelberg:Springer,2012:413-422
[8]Farenzena M,Bazzani L,Perina A,et al.Person re-identification by symmetry-driven accumulation of local features[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2010:2360-2367
[9]Hu Y,Liao S C,Lei Z,et al.Exploring structural information and fusing multiple features for person re-identification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2013:794-799
[10]Liao S C,Hu Y,Zhu X Y,et al.Person re-identification by local maximal occurrence representation and metric learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE,2015:2197-2206
[11]Peng Zhiyong,Chang Faliang,Liu Hongbin,et al.Person re-identification algorithm based on HSV model and keypoints matching[J].Journal of Optoelectronics Laser,2015,26(8):1575-1582(in Chinese)(彭志勇,常发亮,刘洪彬,等.基于HSV模型和特征点匹配的行人重识别算法[J].光电子?激光,2015,26(8):1575-1582)
[12]Weinberger K Q,Saul L K.Distance metric learning for large margin nearest neighbor classification[J].Journal of Machine Learning Research,2006,10:207-244
[13]Shi H L,Yang Y,Zhu X Y,et al.Embedding deep metric for person re-identification:a study against large variations[C]//Proceedings of the 14th European Conference on Computer Vision.Heidelberg:Springer,2016:732-748
[14]Xiao T,Li H S,Ouyang W L,et al.Learning deep feature representations with domain guided dropout for person re-identification[C]//Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:1249-1258
[15]Xu Y L,Ma B P,Huang R,et al.Person search in a scene by jointly modeling people commonness and person uniqueness[C]//Proceedings of the 22nd ACM International Conference on Multimedia.New York:ACM Press,2014:937-940
[16]Zheng L,Zhang H H,Sun S Y,et al.Person re-identification in the wild[C]//Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2017:3346-3355
[17]Xiao T,Li S,Wang B C,et al.End-to-end deep learning for person search[OL].[2017-12-31].http://www.ee.cuhk.edu.hk/~xgwang/PS/paper.pdf
[18]Xiao T,Li S,Wang B C,et al.Joint detection and identification feature learning for person search[C]//Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:3376-3385
[19]Wen Y D,Zhang K P,Li Z F,et al.A discriminative feature learning approach for deep face recognition[C]//Proceedings of the 14th European Conference on Computer Vision.Heidelberg:Springer,2016:499-515
[20]He K M,Zhang X Y,Ren S Q,et al.Deep residual learning for image recognition[C]//Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:770-778
[21]Jia Y Q,Shelhamer E,Donahue J,et al.Caffe:convolutional architecture for fast feature embedding[C]//Proceedings of the22nd ACM International Conference on Multimedia.New York:ACM Press,2014:675-678