款式特征描述符的服装图像细粒度分类方法
|
摘要
针对服装图像大多基于简单款式的粗粒度分类导致分类准确率较低的问题,以款式多样的时尚女装为例,提出一种款式特征描述符的服装图像细粒度分类方法.首先结合时尚女装训练集对输入的待分类时尚女装图像进行部件检测;然后分别提取部件检测后时尚女装图像以及训练图像的HOG, LBP,颜色直方图和边缘算子4种底层特征,得到特征提取后的图像;再将自定义的款式特征描述符与提取到的4种底层特征进行匹配,采用随机森林和多类SVM对时尚女装款式和属性进行监督学习;最后实现时尚女装图像的细粒度分类并输出结果.实验结果表明,该方法能准确地检测并分类出不同服装,提高了服装分类的精度和准确率,能较好地满足实际应用中的需求.
In order to solve the problem of unsatisfied accuracy with the simple styles of coarse-grained clothing image classification, a fine-grained clothing image classification method by style feature description is proposed for various fashion women clothing. Firstly, the part-based detection was conducted to detect the unclassified images combing the training dataset of fashion women clothing. Secondly, four kinds of low-level features including HOG, LBP, color histogram, and edge operator of the training images and the unclassified clothing images after part-based detection were extracted, which can respectively obtain the images after feature extraction. Then, the style feature description was defined to match the four low-level features, and the styles and attributes of fashion women clothing were obtained by the supervised learning using random forests and multi-class SVM. Finally, the fine-grained classification results were implemented and output. Experimental results show that the proposed method can accurately detect and classify various types of clothing images,and the classification accuracy and precision are improved greatly with the better practical applications.
In order to solve the problem of unsatisfied accuracy with the simple styles of coarse-grained clothing image classification, a fine-grained clothing image classification method by style feature description is proposed for various fashion women clothing. Firstly, the part-based detection was conducted to detect the unclassified images combing the training dataset of fashion women clothing. Secondly, four kinds of low-level features including HOG, LBP, color histogram, and edge operator of the training images and the unclassified clothing images after part-based detection were extracted, which can respectively obtain the images after feature extraction. Then, the style feature description was defined to match the four low-level features, and the styles and attributes of fashion women clothing were obtained by the supervised learning using random forests and multi-class SVM. Finally, the fine-grained classification results were implemented and output. Experimental results show that the proposed method can accurately detect and classify various types of clothing images,and the classification accuracy and precision are improved greatly with the better practical applications.
引文
[1]Liu S,Liu L Q,Yan S C.Fashion analysis:current techniques and future directions[J].IEEE MultiMedia,2014,21(2):72-79
[2]Wei Jie,Wu Jigang,Meng Min.Fine-grained image classification with low-dimensional visual feature embedding[J].Journal of Computer-Aided Design&Computer Graphics,2017,29(12):2330-2335(in Chinese)(魏杰,武继刚,孟敏.基于视觉特征低维嵌入的细粒度图像分类[J].计算机辅助设计与图形学学报,2017,29(12):2330-2335)
[3]Luo Jianhao,Wu Jianxin.A survey on fine-grained image categorization using deep convolutional features[J].Acta Automatica Sinica,2017,43(8):1306-1318(in Chinese)(罗建豪,吴建鑫.基于深度卷积特征的细粒度图像分类研究综述[J].自动化学报,2017,43(8):1306-1318)
[4]Bourdev L,Maji S,Malik J.Describing people:a poselet-based approach to attribute classification[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2011:1543-1550
[5]Kalantidis Y,Kennedy L,Li L J.Getting the look:clothing recognition and segmentation for automatic product suggestions in everyday photos[C]//Proceedings of the 3rd ACMConference on International Conference on Multimedia Retrieval.New York:ACM Press,2013:105-112
[6]Lu H Q,Xu C S,Liu G C,et al.Street-to-shop:cross-scenario clothing retrieval via parts alignment and auxiliary set[C]//Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2012:3330-3337
[7]Bossard L,Dantone M,Leistner C,et al.Apparel classification with style[C]//Proceedings of Asian Conference on Computer Vision.Heidelberg:Springer,2012:321-335
[8]Huang J S,Xia W,Yan S C.Deep search with attribute-aware deep network[C]//Proceedings of the 22nd ACM International Conference on Multimedia.New York:ACM Press,2014:731-732
[9]Yamaguchi K,Okatani T,Sudo K,et al.Mix and match:joint model for clothing and attribute recognition[C]//Proceedings of the 26th British Machine Vision Conference.Cambridge:British Machine Vision Association Press,2015:51.1-51.12
[10]Chen Q,Huang J S,Feris R,et al.Deep domain adaptation for describing people based on fine-grained clothing attributes[C]//Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2015:5315-5324
[11]Manfredi M,Grana C,Calderara S,et al.A complete system for garment segmentation and color classification[J].Machine Vision and Applications,2014,25(4):955-969
[12]Yamaguchi K,Hadi Kiapour M,Berg T L.Paper doll parsing:retrieving similar styles to parse clothing items[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2013:3519-3526
[13]Lin K,Yang H F,Liu K H,et al.Rapid clothing retrieval via deep learning of binary codes and hierarchical search[C]//Proceedings of the 5th ACM on International Conference on Multimedia Retrieval.New York:ACM Press,2015:499-502
[14]Hidayati S C,Cheng W H,Hua K L.Clothing genre classification by exploiting the style elements[C]//Proceedings of the20th ACM International Conference on Multimedia.New York:ACM Press,2012:1137-1140
[15]Simo-Serra E,Ishikawa H.Fashion style in 128 floats:joint ranking and classification using weak data for feature extraction[C]//Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2016:298-307
[16]Jaha E S,Nixon M S.Soft biometrics for subject identification using clothing attributes[C]//Proceedings of the IEEE International Joint Conference on Biometrics.Los Alamitos:IEEEComputer Society Press,2014:1-6
[17]Hidayati S C,You C W,Cheng W H,et al.Learning and recognition of clothing genres from full-body images[J].IEEETransactions on Cybernetics,2018,48(5):1647-1659
[18]Di W,Wah C,Bhardwaj A,et al.Style finder:fine-grained clothing style detection and retrieval[C]//Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2013:8-13
[19]Felzenszwalb P F,Huttenlocher D P.Distance transforms of sampled functions[J].Theory of Computing,2012,8(19):415-428
[20]Zhou L,Zhou Z T,Hu D W,Scene classification using a multi-resolution bag-of-features model[J].Pattern Recognition,2013,46(1):424-433
[21]Breiman L.Random forests[J].Machine Learning,2001,45(1):5-32
[22]Rifkin R,Klautau A.In defense of one-Vs-all classification[J].Journal of Machine Learning Research,2004,5(1):101-141
[23]Liu Z W,Luo P,Qiu S,et al.DeepFashion:powering robust clothes recognition and retrieval with rich annotations[C]//Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2016:1096-1104
[24]Wang X W,Zhang T.Clothes search in consumer photos via color matching and attribute learning[C]//Proceedings of the19th ACM International Conference on Multimedia.New York:ACM Press,2011:1353-1356
[25]Chen H Z,Gallagher A,Girod B.Describing clothing by semantic attributes[C]//Proceedings of European Conference on Computer Vision.Heidelberg:Springer,2012:609-623
[26]Kiapour M H,Yamaguchi K,Berg A C,et al.Hipster Wars:discovering elements of fashion styles[C]//Proceedings of European Conference on Computer Vision.Heidelberg:Springer,2014:472-488
[2]Wei Jie,Wu Jigang,Meng Min.Fine-grained image classification with low-dimensional visual feature embedding[J].Journal of Computer-Aided Design&Computer Graphics,2017,29(12):2330-2335(in Chinese)(魏杰,武继刚,孟敏.基于视觉特征低维嵌入的细粒度图像分类[J].计算机辅助设计与图形学学报,2017,29(12):2330-2335)
[3]Luo Jianhao,Wu Jianxin.A survey on fine-grained image categorization using deep convolutional features[J].Acta Automatica Sinica,2017,43(8):1306-1318(in Chinese)(罗建豪,吴建鑫.基于深度卷积特征的细粒度图像分类研究综述[J].自动化学报,2017,43(8):1306-1318)
[4]Bourdev L,Maji S,Malik J.Describing people:a poselet-based approach to attribute classification[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2011:1543-1550
[5]Kalantidis Y,Kennedy L,Li L J.Getting the look:clothing recognition and segmentation for automatic product suggestions in everyday photos[C]//Proceedings of the 3rd ACMConference on International Conference on Multimedia Retrieval.New York:ACM Press,2013:105-112
[6]Lu H Q,Xu C S,Liu G C,et al.Street-to-shop:cross-scenario clothing retrieval via parts alignment and auxiliary set[C]//Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2012:3330-3337
[7]Bossard L,Dantone M,Leistner C,et al.Apparel classification with style[C]//Proceedings of Asian Conference on Computer Vision.Heidelberg:Springer,2012:321-335
[8]Huang J S,Xia W,Yan S C.Deep search with attribute-aware deep network[C]//Proceedings of the 22nd ACM International Conference on Multimedia.New York:ACM Press,2014:731-732
[9]Yamaguchi K,Okatani T,Sudo K,et al.Mix and match:joint model for clothing and attribute recognition[C]//Proceedings of the 26th British Machine Vision Conference.Cambridge:British Machine Vision Association Press,2015:51.1-51.12
[10]Chen Q,Huang J S,Feris R,et al.Deep domain adaptation for describing people based on fine-grained clothing attributes[C]//Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2015:5315-5324
[11]Manfredi M,Grana C,Calderara S,et al.A complete system for garment segmentation and color classification[J].Machine Vision and Applications,2014,25(4):955-969
[12]Yamaguchi K,Hadi Kiapour M,Berg T L.Paper doll parsing:retrieving similar styles to parse clothing items[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2013:3519-3526
[13]Lin K,Yang H F,Liu K H,et al.Rapid clothing retrieval via deep learning of binary codes and hierarchical search[C]//Proceedings of the 5th ACM on International Conference on Multimedia Retrieval.New York:ACM Press,2015:499-502
[14]Hidayati S C,Cheng W H,Hua K L.Clothing genre classification by exploiting the style elements[C]//Proceedings of the20th ACM International Conference on Multimedia.New York:ACM Press,2012:1137-1140
[15]Simo-Serra E,Ishikawa H.Fashion style in 128 floats:joint ranking and classification using weak data for feature extraction[C]//Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2016:298-307
[16]Jaha E S,Nixon M S.Soft biometrics for subject identification using clothing attributes[C]//Proceedings of the IEEE International Joint Conference on Biometrics.Los Alamitos:IEEEComputer Society Press,2014:1-6
[17]Hidayati S C,You C W,Cheng W H,et al.Learning and recognition of clothing genres from full-body images[J].IEEETransactions on Cybernetics,2018,48(5):1647-1659
[18]Di W,Wah C,Bhardwaj A,et al.Style finder:fine-grained clothing style detection and retrieval[C]//Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2013:8-13
[19]Felzenszwalb P F,Huttenlocher D P.Distance transforms of sampled functions[J].Theory of Computing,2012,8(19):415-428
[20]Zhou L,Zhou Z T,Hu D W,Scene classification using a multi-resolution bag-of-features model[J].Pattern Recognition,2013,46(1):424-433
[21]Breiman L.Random forests[J].Machine Learning,2001,45(1):5-32
[22]Rifkin R,Klautau A.In defense of one-Vs-all classification[J].Journal of Machine Learning Research,2004,5(1):101-141
[23]Liu Z W,Luo P,Qiu S,et al.DeepFashion:powering robust clothes recognition and retrieval with rich annotations[C]//Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2016:1096-1104
[24]Wang X W,Zhang T.Clothes search in consumer photos via color matching and attribute learning[C]//Proceedings of the19th ACM International Conference on Multimedia.New York:ACM Press,2011:1353-1356
[25]Chen H Z,Gallagher A,Girod B.Describing clothing by semantic attributes[C]//Proceedings of European Conference on Computer Vision.Heidelberg:Springer,2012:609-623
[26]Kiapour M H,Yamaguchi K,Berg A C,et al.Hipster Wars:discovering elements of fashion styles[C]//Proceedings of European Conference on Computer Vision.Heidelberg:Springer,2014:472-488