聚合CNN特征的遥感图像检索
|
摘要
针对高分辨率遥感图像检索中手工特征难以准确描述图像的问题,提出聚合卷积神经网络(convolutional neural network,CNN)特征的方法来改进特征表达。首先,将预训练的CNN参数迁移到遥感图像,并针对不同尺寸的输入图像,提取表达局部信息的CNN特征;然后,对该CNN特征采用池化区域尺寸不同的均值池化和视觉词袋(bag of visual words,Bo VW) 2种聚合方法,分别得到池化特征和Bo VW特征;最后,将2种聚合特征用于遥感图像检索。实验结果表明:合理的输入图像尺寸能提高聚合特征的表达能力;当池化区域为特征图的60%～80%时,绝大多数池化特征的结果优于传统均值池化方法的结果;池化特征和Bo VW特征的最优平均归一化修改检索等级值比手工特征分别降低了27. 31%和21. 51%,因此,均值池化和Bo VW方法都能有效提高遥感图像的检索性能。
In the high-resolution remote sensing image retrieval,it is difficult for hand-crafted features to describe the images accurately.Thus a method based on aggregating convolutional neural network(CNN) features is proposed to improve the feature representation.First,the parameters from CNN pre-trained on large-scale datasets are transferred for remote sensing images.Given input images with different sizes,the CNN features which represent local information are extracted.Then,average pooling with different pooling region sizes and bag of visual words(BoVW) are adopted to aggregate the CNN features.Pooling features and BoVW features are obtained accordingly.Finally,the above two aggregation features are utilized for remote sensing image retrieval.Experimental results demonstrate that the input image with reasonable size is capable of improving the feature representation.When the pooling region size is between 60% and 80% of the feature map,the vast majority of the results of pooling features are superior to those of the traditional average pooling method.The optimal average normalized modified retrieval rank values of pooling feature and BoVW feature are 27.31% and 21.51% lower than those of hand-crafted feature.Therefore,both the average pooling and BoVW can improve the remote sensing image retrieval performance efficiently.
In the high-resolution remote sensing image retrieval,it is difficult for hand-crafted features to describe the images accurately.Thus a method based on aggregating convolutional neural network(CNN) features is proposed to improve the feature representation.First,the parameters from CNN pre-trained on large-scale datasets are transferred for remote sensing images.Given input images with different sizes,the CNN features which represent local information are extracted.Then,average pooling with different pooling region sizes and bag of visual words(BoVW) are adopted to aggregate the CNN features.Pooling features and BoVW features are obtained accordingly.Finally,the above two aggregation features are utilized for remote sensing image retrieval.Experimental results demonstrate that the input image with reasonable size is capable of improving the feature representation.When the pooling region size is between 60% and 80% of the feature map,the vast majority of the results of pooling features are superior to those of the traditional average pooling method.The optimal average normalized modified retrieval rank values of pooling feature and BoVW feature are 27.31% and 21.51% lower than those of hand-crafted feature.Therefore,both the average pooling and BoVW can improve the remote sensing image retrieval performance efficiently.
引文
[1]朱佳丽,李士进,万定生,等.基于特征选择和半监督学习的遥感图像检索[J].中国图象图形学报,2011,16(8):1474-1482.Zhu J L,Li S J,Wan D S,et al.Content-based remote sensing image retrieval based on feature selection and semi-supervised learning[J].Journal of Image and Graphics,2011,16(8):1474-1482.
[2]Demir B,Bruzzone L.A novel active learning method in relevance feedback for content-based remote sensing image retrieval[J].IEEE Transactions on Geoscience and Remote Sensing,2015,53(5):2323-2334.
[3]Aptoula E.Remote sensing image retrieval with global morphological texture descriptors[J].IEEE Transactions on Geoscience and Remote Sensing,2014,52(5):3023-3034.
[4]陆丽珍,刘仁义,刘南.一种融合颜色和纹理特征的遥感图像检索方法[J].中国图象图形学报,2004,9(3):328-333.Lu L Z,Liu R Y,Liu N.Remote sensing image retrieval using color and texture fused features[J].Journal of Image and Graphics,2004,9(3):328-333.
[5]Yang Y,Newsam S.Geographic image retrieval using local invariant features[J].IEEE Transactions on Geoscience and Remote Sensing,2013,51(2):818-832.
[6]Du Z X,Li X L,Lu X Q.Local structure learning in high resolution remote sensing image retrieval[J].Neurocomputing,2016(207):813-822.
[7]Babenko A,Slesarev A,Chigorin A,et al.Neural codes for image retrieval[C]//Proceedings of European Conference on Computer Vision.Springer,2014:584-599.
[8]Ng J Y,Yang F,Davis L S.Exploiting local features from deep networks for image[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition workshops.IEEE,2015:53-61.
[9]Babenko A,Lempitsky V.Aggregating deep convolutional features for image retrieval[C]//Proceedings of IEEE International Conference on Computer Vision.IEEE,2015:1269-1277.
[10]周飞燕,金林鹏,董军.卷积神经网络研究综述[J].计算机学报,2017,40(6):1229-1251.Zhou F Y,Jin L P,Dong J.Review of convolutional neural network[J].Chinese Journal of Computers,2017,40(6):1229-1251.
[11]张洪群,刘雪莹,杨森,等.深度学习的半监督遥感图像检索[J].遥感学报,2017,21(3):406-414.Zhang H Q,Liu X Y,Yang S,et al.Retrieval of remote sensing images based on semisupervised deep learning[J].Journal of Remote Sensing,21(3):406-414.
[12]Napoletano P.Visual descriptors for content-based retrieval of remote sensing images[J].International Journal of Remote Sensing,2018,39(5):1343-1376.
[13]Zhou W X,Newsam S,Li C,et al.Learning low dimensional convolutional neural networks for high-resolution remote sensing image retrieval[J].Remote Sensing,2017,9(5):489.
[14]Hu F,Tong X Y,Xia G S,et al.Delving into deep representations for remote sensing image retrieval[C]//Proceedings of IEEE International Conference on Signal Processing.IEEE,2016:198-203.
[15]Simonyan K,Zisserman A.Very deep convolutional networks for large-scale image recognition[EB/OL].ar Xiv.https://arxiv.org/pdf/1409.1556.pdf.
[16]Szegedy C,Liu W,Jia Y Q,et al.Going deeper with convolutions[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2015:1-9.
[17]Hu F,Xia G S,Hu J W,et al.Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery[J].Remote Sensing,2015,7(11):14680-14707.
[18]Vedaldi A,Lenc K.Mat ConvNet:convolutional neural networks for MATLAB[C]//Proceedings of 23rd ACM International Conference on Multimedia.ACM,2015:689-692.
[2]Demir B,Bruzzone L.A novel active learning method in relevance feedback for content-based remote sensing image retrieval[J].IEEE Transactions on Geoscience and Remote Sensing,2015,53(5):2323-2334.
[3]Aptoula E.Remote sensing image retrieval with global morphological texture descriptors[J].IEEE Transactions on Geoscience and Remote Sensing,2014,52(5):3023-3034.
[4]陆丽珍,刘仁义,刘南.一种融合颜色和纹理特征的遥感图像检索方法[J].中国图象图形学报,2004,9(3):328-333.Lu L Z,Liu R Y,Liu N.Remote sensing image retrieval using color and texture fused features[J].Journal of Image and Graphics,2004,9(3):328-333.
[5]Yang Y,Newsam S.Geographic image retrieval using local invariant features[J].IEEE Transactions on Geoscience and Remote Sensing,2013,51(2):818-832.
[6]Du Z X,Li X L,Lu X Q.Local structure learning in high resolution remote sensing image retrieval[J].Neurocomputing,2016(207):813-822.
[7]Babenko A,Slesarev A,Chigorin A,et al.Neural codes for image retrieval[C]//Proceedings of European Conference on Computer Vision.Springer,2014:584-599.
[8]Ng J Y,Yang F,Davis L S.Exploiting local features from deep networks for image[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition workshops.IEEE,2015:53-61.
[9]Babenko A,Lempitsky V.Aggregating deep convolutional features for image retrieval[C]//Proceedings of IEEE International Conference on Computer Vision.IEEE,2015:1269-1277.
[10]周飞燕,金林鹏,董军.卷积神经网络研究综述[J].计算机学报,2017,40(6):1229-1251.Zhou F Y,Jin L P,Dong J.Review of convolutional neural network[J].Chinese Journal of Computers,2017,40(6):1229-1251.
[11]张洪群,刘雪莹,杨森,等.深度学习的半监督遥感图像检索[J].遥感学报,2017,21(3):406-414.Zhang H Q,Liu X Y,Yang S,et al.Retrieval of remote sensing images based on semisupervised deep learning[J].Journal of Remote Sensing,21(3):406-414.
[12]Napoletano P.Visual descriptors for content-based retrieval of remote sensing images[J].International Journal of Remote Sensing,2018,39(5):1343-1376.
[13]Zhou W X,Newsam S,Li C,et al.Learning low dimensional convolutional neural networks for high-resolution remote sensing image retrieval[J].Remote Sensing,2017,9(5):489.
[14]Hu F,Tong X Y,Xia G S,et al.Delving into deep representations for remote sensing image retrieval[C]//Proceedings of IEEE International Conference on Signal Processing.IEEE,2016:198-203.
[15]Simonyan K,Zisserman A.Very deep convolutional networks for large-scale image recognition[EB/OL].ar Xiv.https://arxiv.org/pdf/1409.1556.pdf.
[16]Szegedy C,Liu W,Jia Y Q,et al.Going deeper with convolutions[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2015:1-9.
[17]Hu F,Xia G S,Hu J W,et al.Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery[J].Remote Sensing,2015,7(11):14680-14707.
[18]Vedaldi A,Lenc K.Mat ConvNet:convolutional neural networks for MATLAB[C]//Proceedings of 23rd ACM International Conference on Multimedia.ACM,2015:689-692.