结合卷积神经网络多层特征融合和K-Means聚类的服装图像检索方法
|
摘要
随着服装电子商务的蓬勃发展,海量的服装图像数据被累积,对服装图像"以图搜图"成为了当前的一个热点研究方向。服装图像有着丰富的整体语义信息和大量细节信息,要对其实现精准检索是一项挑战性难题。传统的基于人工语义标注的服装图像方法和以人工设计的颜色与纹理等内容特征进行服装图像检索的方法均存在较大局限性。文中利用卷积神经网络多层特征融合提取特征,然后使用K-Means聚类加快服装图像的检索,充分利用深度卷积神经网络在图像特征提取上的有效性和层次性,融合不同卷积层次特征的细节信息和抽象语义信息以提升检索的准确度,并利用K-Means加快检索速度。所提方法首先对服装图像数据集进行统一的尺寸处理,然后利用卷积神经网络进行训练和特征提取,抽取出服装图像从低到高的多层次特征,进而将多种层次的特征进行融合,最终使用K-Means聚类方法对提取的图像库特征进行有效检索。在DeepFashion子类数据集Category and Attribute Prediction Benchmark和In-shop Clothes Retrieval Benchmark上的实验结果表明,所提方法能有效增强服装图像的特征表达能力,提高了检索准确率和检索速度,优于其他主流方法。
The booming of clothing e-commerce has accumulated a large amount of clothing image data,and the "image search" of clothing images has become a hot research direction.Apparel images have rich overall semantic information and a large amount of detailed information,and achieving accurate retrieval is a challenging problem.Traditional me-thods of clothing image based on artificial semantic annotation and methods of image retrieval based on artificially designed content features such as color and texture have significant limitations.This paper proposed a clothing image retrieval method based on multi-layer feature fusion and K-Means clustering of convolutional neural networks,which makes full use of the effectiveness and hierarchy of deep convolutional neural network in image feature extraction,fuses the detailed information and abstract semantic information of different convolutional hierarchical features to improve retrieval accuracy,and uses K-Means to improve the retrieval speed.The proposed method firstly performs uniform size processing on the clothing image data set,then uses the convolutional neural network for training and feature extraction,extracts multi-level features of the clothing image from low to high,and then fuses various levels of features.Finally,the K-Means clustering method is used to efficiently retrieve large-scale image data.The experimental results on the DeepFashion sub-category data set Category and Attribute Prediction Benchmark and In-shop Clothes Retrieval Benchmark show that the proposed method can effectively enhance the feature expression ability of clothing images,and improve its retrieval accuracy and retrieval speed.The proposed method is supprior to other mainstream methods.
The booming of clothing e-commerce has accumulated a large amount of clothing image data,and the "image search" of clothing images has become a hot research direction.Apparel images have rich overall semantic information and a large amount of detailed information,and achieving accurate retrieval is a challenging problem.Traditional me-thods of clothing image based on artificial semantic annotation and methods of image retrieval based on artificially designed content features such as color and texture have significant limitations.This paper proposed a clothing image retrieval method based on multi-layer feature fusion and K-Means clustering of convolutional neural networks,which makes full use of the effectiveness and hierarchy of deep convolutional neural network in image feature extraction,fuses the detailed information and abstract semantic information of different convolutional hierarchical features to improve retrieval accuracy,and uses K-Means to improve the retrieval speed.The proposed method firstly performs uniform size processing on the clothing image data set,then uses the convolutional neural network for training and feature extraction,extracts multi-level features of the clothing image from low to high,and then fuses various levels of features.Finally,the K-Means clustering method is used to efficiently retrieve large-scale image data.The experimental results on the DeepFashion sub-category data set Category and Attribute Prediction Benchmark and In-shop Clothes Retrieval Benchmark show that the proposed method can effectively enhance the feature expression ability of clothing images,and improve its retrieval accuracy and retrieval speed.The proposed method is supprior to other mainstream methods.
引文
[1] ALBIOL A,MONZO D,MARTIN A,et al.Face recognition using HOG-EBGM[J].Pattern Recognition Letters,2008,29(10):1537-1543.
[2] LO T W R,SIEBERT J P.Local feature extraction and matching on range images:2.5D SIFT[J].Computer Vision & Image Understanding,2009,113(12):1235-1250.
[3] ZHOU L,GENG Z,ZHANG J,et al.ORB feature based web pornographic image recognition[J].Neurocomputing,2016,173(P3):511-517.
[4] 贾巧丽,王娟,孔兵.基于形状特征和颜色的服装图像检索[J].现代计算机(专业版),2011(7):30-32.
[5] 薛培培,邬延辉.基于图像内容和支持向量机的服装图像检索方法研究[J].移动通信,2016(2):79-82.
[6] 胡玉平,肖行,罗东俊.基于GrabCut改进算法的服装图像检索方法[J].计算机科学,2016,43(S2):242-246.
[7] HINTON G,OSINDERO S.A fast learning algorithm for deep belief nets [J].Neural Computation,2006,18(7):1527-1554.
[8] CIRESAN D,MEIER U,SCHMIDHUBER J.Multi-column Deep Neural Networks for Image Classification [C]//Procee-dings of IEEE Conference on Computer Vision and Pattern Re-cognition.Washington D C,USA:IEEE Press,2012:3642-3649.
[9] CIRSHICK R,DONAHUE J,DARRELL T,et al.Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation [C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D C,USA:IEEE Press,2014:580-587.
[10] LIN K,YANG H F,LIU K H,et al.Rapid clothing retrieval via deep learning of binary codes and hierarchical search[C]//Proceedings of the 5th ACM on International Conference on Multimedia Retrieval.ACM,2015:498-502.
[11] KRIZHECSY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems,2012,25(2):1097-1105.
[12] KIAPOU M H,HAN X,LAZEBNIK S,et al.Where to Buy It:Matching Street Clothing Photos in Online Shopes[C]//2015 IEEE International Conference on Computer Vision(ICCV).Santiago:IEEE,2015:3343-3351.
[13] FUKUSHIMA K.Neocognitron:a self-organizing neural net- work model for a mechanism of pattern recognition unaffected by shift in position [J].Biological Cybernetics,1980,36(4):193-202.
[14] 王利华,邹俊忠,张见,等.基于深度卷积神经网络的快速图像分类算法[J].计算机工程与应用,2017,53(13):181-188.
[15] 刘海龙,李宝安,吕学强,等.基于深度卷积神经网络的图像检索算法研究[J].计算机应用研究,2017,34(12):3816-3819.
[16] YIM J,JU J,JUNG H,et al.Image Classification Using Convolutional Neural Networks With Multi-stage Feature [M]//Robot Intelligence Technology and Applications 3.Springer International Publishing,2015.
[17] SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Boston:IEEE,2015.
[18] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition[J].Computer Science,2014,1(2):3.
[19] HE K,ZHANG X,REN S,et al.Deep Residual Learning for Ima- ge Recognition [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE Computer Society,2016:770-778.
[20] SCHROFF F,KALENICHENKO D,PHILBIN J.FaceNet:A unified embedding for face recognition and clustering[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Boston:IEEE,2015:815-823.
[21] SURAL S,QIAN G,PRAMANIK S.Segmentation and histogram generation using the HSV color space for image retrieval[C]//International Conference on Image Processing.IEEE,2002:589-592.
[22] DALAL N,TRIGGS B.Histograms of oriented gradients for human detection[C]//IEEE Computer Society Conference on Computer Vision and Pattern Recognition.IEEE,2005:886-893.
[23] CHANG C C,LIN C J.LIBSVM:A library for support vector machines[J].ACM Transactions on Intelligent Systems and Technology,2011,2(3):1-27.
[2] LO T W R,SIEBERT J P.Local feature extraction and matching on range images:2.5D SIFT[J].Computer Vision & Image Understanding,2009,113(12):1235-1250.
[3] ZHOU L,GENG Z,ZHANG J,et al.ORB feature based web pornographic image recognition[J].Neurocomputing,2016,173(P3):511-517.
[4] 贾巧丽,王娟,孔兵.基于形状特征和颜色的服装图像检索[J].现代计算机(专业版),2011(7):30-32.
[5] 薛培培,邬延辉.基于图像内容和支持向量机的服装图像检索方法研究[J].移动通信,2016(2):79-82.
[6] 胡玉平,肖行,罗东俊.基于GrabCut改进算法的服装图像检索方法[J].计算机科学,2016,43(S2):242-246.
[7] HINTON G,OSINDERO S.A fast learning algorithm for deep belief nets [J].Neural Computation,2006,18(7):1527-1554.
[8] CIRESAN D,MEIER U,SCHMIDHUBER J.Multi-column Deep Neural Networks for Image Classification [C]//Procee-dings of IEEE Conference on Computer Vision and Pattern Re-cognition.Washington D C,USA:IEEE Press,2012:3642-3649.
[9] CIRSHICK R,DONAHUE J,DARRELL T,et al.Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation [C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D C,USA:IEEE Press,2014:580-587.
[10] LIN K,YANG H F,LIU K H,et al.Rapid clothing retrieval via deep learning of binary codes and hierarchical search[C]//Proceedings of the 5th ACM on International Conference on Multimedia Retrieval.ACM,2015:498-502.
[11] KRIZHECSY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems,2012,25(2):1097-1105.
[12] KIAPOU M H,HAN X,LAZEBNIK S,et al.Where to Buy It:Matching Street Clothing Photos in Online Shopes[C]//2015 IEEE International Conference on Computer Vision(ICCV).Santiago:IEEE,2015:3343-3351.
[13] FUKUSHIMA K.Neocognitron:a self-organizing neural net- work model for a mechanism of pattern recognition unaffected by shift in position [J].Biological Cybernetics,1980,36(4):193-202.
[14] 王利华,邹俊忠,张见,等.基于深度卷积神经网络的快速图像分类算法[J].计算机工程与应用,2017,53(13):181-188.
[15] 刘海龙,李宝安,吕学强,等.基于深度卷积神经网络的图像检索算法研究[J].计算机应用研究,2017,34(12):3816-3819.
[16] YIM J,JU J,JUNG H,et al.Image Classification Using Convolutional Neural Networks With Multi-stage Feature [M]//Robot Intelligence Technology and Applications 3.Springer International Publishing,2015.
[17] SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Boston:IEEE,2015.
[18] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition[J].Computer Science,2014,1(2):3.
[19] HE K,ZHANG X,REN S,et al.Deep Residual Learning for Ima- ge Recognition [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE Computer Society,2016:770-778.
[20] SCHROFF F,KALENICHENKO D,PHILBIN J.FaceNet:A unified embedding for face recognition and clustering[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Boston:IEEE,2015:815-823.
[21] SURAL S,QIAN G,PRAMANIK S.Segmentation and histogram generation using the HSV color space for image retrieval[C]//International Conference on Image Processing.IEEE,2002:589-592.
[22] DALAL N,TRIGGS B.Histograms of oriented gradients for human detection[C]//IEEE Computer Society Conference on Computer Vision and Pattern Recognition.IEEE,2005:886-893.
[23] CHANG C C,LIN C J.LIBSVM:A library for support vector machines[J].ACM Transactions on Intelligent Systems and Technology,2011,2(3):1-27.