基于残差网络的三维模型检索算法
|
摘要
近年来,基于视图的3D模型检索已经成为计算机视觉领域的重点研究方向。3D模型检索算法包括特征提取和检索算法两个部分,且鲁棒的特征对于检索算法起着决定性的作用。目前,研究者们已经提出了许多人工设计特征和深度学习特征,但是很少有人比较它们的异同。因此,文中对不同的人工设计特征和深度学习特征的性能进行了评估分析,基于充分对比的前提,采用了多个数据集、多样的评价标准和不同的检索算法进行了实验,并进一步比较了深度网络不同层特征对性能的影响,从而提出了基于残差网络的三维模型检索算法。在多个公开数据集上的实验表明:1)残差网络所提取的深度特征相较于传统特征,综合性能提升了1%～20%;2)与VGG网络所提取的深度特征相比,残差网络的综合性能提升了1%～5%;3)VGG网络中不同层特征的性能也有差异,深层特征与浅层特征相比,综合性能提升了1%～6%;4)随着网络深度的增加,残差网络所提取的特征的综合性能得到了有限提高,且比其他对比特征均更加鲁棒。
In recent years,view-based 3 D model retrieval has become a key research direction in the field of computer vision.The 3 D model retrieval algorithm includes feature extraction and model retrieval where robust features play a decisive role in retrieval algorithm.Up to now,the traditional hand-crafted features and deep learning features were proposed,but very few people systematically compare them.Therefore,in this work,the performance of different artificial design features and deep learning features was evaluated and analyzed.Based on the premise of full comparison,multiple data sets,multiple evaluation criteria,and different search algorithms were used to conduct experiments.The effects of different layers of deep network on performance were further compared,and a 3 D model retrieval algorithm based on residual network was proposed.Several conclusions could be obtained from the experimental results on multiple public datasets.1)When comparing the deep learning features of VGG network and residual network with traditional hand-crafted features,the improvement of comprehensive performance can reaches 3% to 20%.2)Compared with the deep features extracted by VGG network,the comprehensive performance of the residual network is increased by 1% to 5%.3)The performance of different layer features in the VGG network is also different,and the comprehensive performance of the deep and shallow features is increased by 1% to 6%.4)As the depth of the network increase,the overall perfor-mance of the extracted features of the residual network has limited improvement,and is more robust than other contrasting features.
In recent years,view-based 3 D model retrieval has become a key research direction in the field of computer vision.The 3 D model retrieval algorithm includes feature extraction and model retrieval where robust features play a decisive role in retrieval algorithm.Up to now,the traditional hand-crafted features and deep learning features were proposed,but very few people systematically compare them.Therefore,in this work,the performance of different artificial design features and deep learning features was evaluated and analyzed.Based on the premise of full comparison,multiple data sets,multiple evaluation criteria,and different search algorithms were used to conduct experiments.The effects of different layers of deep network on performance were further compared,and a 3 D model retrieval algorithm based on residual network was proposed.Several conclusions could be obtained from the experimental results on multiple public datasets.1)When comparing the deep learning features of VGG network and residual network with traditional hand-crafted features,the improvement of comprehensive performance can reaches 3% to 20%.2)Compared with the deep features extracted by VGG network,the comprehensive performance of the residual network is increased by 1% to 5%.3)The performance of different layer features in the VGG network is also different,and the comprehensive performance of the deep and shallow features is increased by 1% to 6%.4)As the depth of the network increase,the overall perfor-mance of the extracted features of the residual network has limited improvement,and is more robust than other contrasting features.
引文
[1] OHBUCHI R,FURUYA T.Accelerating bag-of-features sift algorithm for 3d model retrieval[C]//Proc. SAMT 2008 Workshop on Semantic 3D Media.2008:23-30.
[2] GAO Y,DAI Q,ZHANG N Y.3D model comparison using spatial structure circular descriptor[J].Pattern Recognition,2010,43(3):1142-1151.
[3] LI B,JOHAN H.3D model retrieval using hybrid features and class information[M].Kluwer Academic Publishers,2013.
[4] VRANIC D V,SAUPE D.3D model retrieval[D].Leipzig:University of Leipzig,2004.
[5] DARAS P,AXENOPOULOS A.A 3D Shape Retrieval Framework Supporting Multimodal Queries[J].International Journal of Computer Vision,2010,89(2-3):229-247.
[6] SHIH J L,LEE C H,WANG J T.A new 3D model retrieval approach based on the elevation descriptor[J].Pattern Recognition,2007,40(1):283-295.
[7] LU K,HE N,XUE J,et al.Learning View-Model Joint Relevance for 3D Object Retrieval[J].IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society,2015,24(5):1449-1459.
[8] OHBUCHI R,FURUYA T.Scale-weighted dense bag of visual features for 3D model retrieval from a partial view 3D model[C]//IEEE,International Conference on Computer Vision Workshops.IEEE,2009:63-70.
[9] DUBUISSON M P,JAIN A K.A modified Hausdorff distance for object matching[C]//IEEE Conference on Computer Vision and Image Processing.1994:566-568.
[10] GAO Y,DAI Q.View-based 3-D object retrieval[M].Holland:Morgan Kaufmann,2014.
[11] ANSARY T F,DAOUDI M,VANDEBORRE J P.A Bayesian 3-D Search Engine Using Adaptive Views Clustering[J].IEEE Transactions on Multimedia,2006,9(1):78-88.
[12] ANSARY T F,DAOUDI M,VANDEBORRE J P.A Bayesian 3-D Search Engine Using Adaptive Views Clustering[J].IEEE Transactions on Multimedia,2006,9(1):78-88.
[13] GAO Y,TANG J,HONG R,et al.Camera Constraint-Free View-Based 3-D Object Retrieval[J].IEEE Transactions on Ima-ge Processing,2012,21(4):2269-2281.
[14] LIU X,WANG M,YIN B C,et al.Event-Based Media Enrichment Using an Adaptive Probabilistic Hypergraph Model.[J].IEEE Transactions on Cybernetics,2015,45(11):2461.
[15] ZHU L,SHEN J,JIN H,et al.Content-Based Visual Landmark Search via Multimodal Hypergraph Learning[J].IEEE Transactions on Cybernetics,2015,45(12):2756.
[16] LIU A A,NIE W Z,GAO Y,et al.Multi-Modal Clique-Graph Matching for View-Based 3D Model Retrieval[J].IEEE Tran-sactions on Image Processing,2016,25(5):2103-2116.
[17] LU K,HE N,XUE J,et al.Learning View-Model Joint Rele- vance for 3D Object Retrieval[J].IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society,2015,24(5):1449-1459.
[18] GAO Y,DAI Q,WANG M,et al.3D model retrieval using weighted bipartite graph matching[J].Signal Processing Image Communication,2011,26(1):39-47.
[19] LEORDEANU M,HEBERT M.A Spectral Technique for Correspondence Problems Using Pairwise Constraints[C]//Tenth IEEE International Conference on Computer Vision.IEEE Computer Society,2005:1482-1489.
[20] CHO M,LEE J,LEE K M.Reweighted random walks for graph matching[C]//European Conference on Computer Vision.Springer,Berlin,Heidelberg,2010:492-505.
[21] LIU A,WANG Z,NIE W,et al.Graph-based characteristic view set extraction and matching for 3D model retrieval[J].Information Sciences,2015,320:429-442.
[22] KHOTANZAD A,HONG Y H.Invariant Image Recognition by Zernike Moments[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,1990,12(5):489-497.
[23] DALAL N,TRIGGS B.Histograms of oriented gradients for human detection[C]//IEEE Computer Society Conference on Computer Vision and Pattern Recognition,2005(CVPR 2005).IEEE,2005:886-893.
[24] PERRONNIN F,MENSINK T.Improving the fisher kernel for large-scale image classification[C]//European Conference on Computer Vision.Springer-Verlag,2010:143-156.
[25] DONAHUE J,JIA Y,VINYALS O,et al.DeCAF:A Deep Convolutional Activation Feature for Generic Visual Recognition[C]∥International Conference on Machine Learning.2014:647-655.
[26] NIE W,CAO Q,LIU A,et al.Convolutional deep learning for 3D object retrieval[J].Multimedia Systems,2017,23(3):325-332.
[27] LEIBE B,SCHIELE B.Analyzing appearance and contour based methods for object categorization[C]//2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.IEEE,2003,2:II-409.
[28] CHEN D Y.On visual similarity based 3d model retrieval[C]//Computer Graphics Forum.2003:223-232.
[29] GAO Y,WANG M,JI R,et al.3-D Object Retrieval With Haus- dorff Distance Learning[J].IEEE Transactions on Industrial Electronics,2013,61(4):2088-2098.
[30] GAO Y,WANG M,ZHA Z J,et al.Less is More:Efficient 3-D Object Retrieval With Query View Selection[J].IEEE Transactions on Multimedia,2011,13(5):1007-1018.
[31] LLER H,LLER W,SQUIRE D M,et al.Performance evaluation in content-based image retrieval:overview and proposals[J].Pattern Recognition Letters,2001,22(5):593-601.
[32] NIE W Z,LIU A A,SU Y T.3D object retrieval based on sparse coding in weak supervision[J].Journal of Visual Communication &Image Representation,2016,37(C):40-45.
[2] GAO Y,DAI Q,ZHANG N Y.3D model comparison using spatial structure circular descriptor[J].Pattern Recognition,2010,43(3):1142-1151.
[3] LI B,JOHAN H.3D model retrieval using hybrid features and class information[M].Kluwer Academic Publishers,2013.
[4] VRANIC D V,SAUPE D.3D model retrieval[D].Leipzig:University of Leipzig,2004.
[5] DARAS P,AXENOPOULOS A.A 3D Shape Retrieval Framework Supporting Multimodal Queries[J].International Journal of Computer Vision,2010,89(2-3):229-247.
[6] SHIH J L,LEE C H,WANG J T.A new 3D model retrieval approach based on the elevation descriptor[J].Pattern Recognition,2007,40(1):283-295.
[7] LU K,HE N,XUE J,et al.Learning View-Model Joint Relevance for 3D Object Retrieval[J].IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society,2015,24(5):1449-1459.
[8] OHBUCHI R,FURUYA T.Scale-weighted dense bag of visual features for 3D model retrieval from a partial view 3D model[C]//IEEE,International Conference on Computer Vision Workshops.IEEE,2009:63-70.
[9] DUBUISSON M P,JAIN A K.A modified Hausdorff distance for object matching[C]//IEEE Conference on Computer Vision and Image Processing.1994:566-568.
[10] GAO Y,DAI Q.View-based 3-D object retrieval[M].Holland:Morgan Kaufmann,2014.
[11] ANSARY T F,DAOUDI M,VANDEBORRE J P.A Bayesian 3-D Search Engine Using Adaptive Views Clustering[J].IEEE Transactions on Multimedia,2006,9(1):78-88.
[12] ANSARY T F,DAOUDI M,VANDEBORRE J P.A Bayesian 3-D Search Engine Using Adaptive Views Clustering[J].IEEE Transactions on Multimedia,2006,9(1):78-88.
[13] GAO Y,TANG J,HONG R,et al.Camera Constraint-Free View-Based 3-D Object Retrieval[J].IEEE Transactions on Ima-ge Processing,2012,21(4):2269-2281.
[14] LIU X,WANG M,YIN B C,et al.Event-Based Media Enrichment Using an Adaptive Probabilistic Hypergraph Model.[J].IEEE Transactions on Cybernetics,2015,45(11):2461.
[15] ZHU L,SHEN J,JIN H,et al.Content-Based Visual Landmark Search via Multimodal Hypergraph Learning[J].IEEE Transactions on Cybernetics,2015,45(12):2756.
[16] LIU A A,NIE W Z,GAO Y,et al.Multi-Modal Clique-Graph Matching for View-Based 3D Model Retrieval[J].IEEE Tran-sactions on Image Processing,2016,25(5):2103-2116.
[17] LU K,HE N,XUE J,et al.Learning View-Model Joint Rele- vance for 3D Object Retrieval[J].IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society,2015,24(5):1449-1459.
[18] GAO Y,DAI Q,WANG M,et al.3D model retrieval using weighted bipartite graph matching[J].Signal Processing Image Communication,2011,26(1):39-47.
[19] LEORDEANU M,HEBERT M.A Spectral Technique for Correspondence Problems Using Pairwise Constraints[C]//Tenth IEEE International Conference on Computer Vision.IEEE Computer Society,2005:1482-1489.
[20] CHO M,LEE J,LEE K M.Reweighted random walks for graph matching[C]//European Conference on Computer Vision.Springer,Berlin,Heidelberg,2010:492-505.
[21] LIU A,WANG Z,NIE W,et al.Graph-based characteristic view set extraction and matching for 3D model retrieval[J].Information Sciences,2015,320:429-442.
[22] KHOTANZAD A,HONG Y H.Invariant Image Recognition by Zernike Moments[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,1990,12(5):489-497.
[23] DALAL N,TRIGGS B.Histograms of oriented gradients for human detection[C]//IEEE Computer Society Conference on Computer Vision and Pattern Recognition,2005(CVPR 2005).IEEE,2005:886-893.
[24] PERRONNIN F,MENSINK T.Improving the fisher kernel for large-scale image classification[C]//European Conference on Computer Vision.Springer-Verlag,2010:143-156.
[25] DONAHUE J,JIA Y,VINYALS O,et al.DeCAF:A Deep Convolutional Activation Feature for Generic Visual Recognition[C]∥International Conference on Machine Learning.2014:647-655.
[26] NIE W,CAO Q,LIU A,et al.Convolutional deep learning for 3D object retrieval[J].Multimedia Systems,2017,23(3):325-332.
[27] LEIBE B,SCHIELE B.Analyzing appearance and contour based methods for object categorization[C]//2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.IEEE,2003,2:II-409.
[28] CHEN D Y.On visual similarity based 3d model retrieval[C]//Computer Graphics Forum.2003:223-232.
[29] GAO Y,WANG M,JI R,et al.3-D Object Retrieval With Haus- dorff Distance Learning[J].IEEE Transactions on Industrial Electronics,2013,61(4):2088-2098.
[30] GAO Y,WANG M,ZHA Z J,et al.Less is More:Efficient 3-D Object Retrieval With Query View Selection[J].IEEE Transactions on Multimedia,2011,13(5):1007-1018.
[31] LLER H,LLER W,SQUIRE D M,et al.Performance evaluation in content-based image retrieval:overview and proposals[J].Pattern Recognition Letters,2001,22(5):593-601.
[32] NIE W Z,LIU A A,SU Y T.3D object retrieval based on sparse coding in weak supervision[J].Journal of Visual Communication &Image Representation,2016,37(C):40-45.