融合全局和局部深度特征的高分辨率遥感影像场景分类方法
|
摘要
提出了一种融合全局和局部深度特征(GLDFB)的视觉词袋模型。通过视觉词袋模型将深度卷积神经网络提取的多个层次的高层特征进行重组编码并融合,利用支持向量机对融合特征进行分类。充分利用包含场景局部细节信息的卷积层特征和包含场景全局信息的全连接层特征,完成对遥感影像场景的高效表达。通过对两个不同规模的遥感图像场景数据集的实验研究表明,相比现有方法,所提方法在高层特征表达能力和分类精度方面具有显著优势。
A global and local deep feature based(GLDFB) bag-of-visual-words(BoVW) model is proposed. The high-level features extracted from the deep convolutional neural network are reorganized and encoded by the BoVW model and the fusion features are classified by the support vector machine. The features from the convolutional layer containing the local details and the fully-connected layer containing the global information of scenes are fully used and thus the efficient expressions of the remote sensing image scenes are formed. The experimental results on two remote sensing image scene datasets with different scales show that, compared with the existing methods, the proposed method possesses unique advantages in the representation ability and the classification accuracy of high-level features.
A global and local deep feature based(GLDFB) bag-of-visual-words(BoVW) model is proposed. The high-level features extracted from the deep convolutional neural network are reorganized and encoded by the BoVW model and the fusion features are classified by the support vector machine. The features from the convolutional layer containing the local details and the fully-connected layer containing the global information of scenes are fully used and thus the efficient expressions of the remote sensing image scenes are formed. The experimental results on two remote sensing image scene datasets with different scales show that, compared with the existing methods, the proposed method possesses unique advantages in the representation ability and the classification accuracy of high-level features.
引文
[1] Cheriyadat A M. Unsupervised feature learning for aerial scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2014, 52(1): 439-451.
[2] Csurka G. Visual categorization with bags of keypoints[C]. European Conference on Computer Vision, 2004: 1-22.
[3] Lazebnik S, Schmid C, Ponce J. Beyond bags of features: spatial pyramid matching for recognizing natural scene categories[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2006: 2169-2178.
[4] Yang Y, Newsam S. Spatial pyramid co-occurrence for image classification[C]. International Conference on Computer Vision, 2011: 1465-1472.
[5] Fang X, Wang G H, Yang H C, et al. High resolution remote sensing image classification combining with mean-shift segmentation and fully convolution neural network[J]. Laser & Optoelectronics Progress, 2018, 55(2): 022802. 方旭, 王光辉, 杨化超, 等. 结合均值漂移分割与全卷积神经网络的高分辨遥感影像分类[J]. 激光与光电子学进展, 2018, 55(2): 022802.
[6] Liu D W, Han L, Han X Y. High spatial resolution remote sensing image classification based on deep learning[J]. Acta Optica Sinica, 2016, 36(4): 0428001. 刘大伟, 韩玲, 韩晓勇. 基于深度学习的高分辨率遥感影像分类研究[J]. 光学学报, 2016, 36(4): 0428001.
[7] Liu F, Lu L X, Huang G W, et al. Landform image classification based on discrete cosine transformation and deep network [J]. Acta Optica Sinica, 2018, 38(6): 0620001. 刘芳, 路丽霞, 黄光伟, 等. 基于离散余弦变换和深度网络的地貌图像分类[J]. 光学学报, 2018, 38(6): 0620001.
[8] Castelluccio M, Poggi G, Sansone C, et al. Land use classification in remote sensing images by convolutional neural networks[J]. Acta Ecologica Sinica, 2015, 28(2): 627-635.
[9] Zhong Y F, Fei F, Zhang L P. Large patch convolutional neural networks for the scene classification of high spatial resolution imagery[J]. Journal of Applied Remote Sensing, 2016, 10(2): 025006.
[10] Chen Y, Fan R S, Wang J X, et al. High resolution image classification method combining with minimum noise fraction rotation and convolution neural network[J]. Laser & Optoelectronics Progress, 2017, 54(10): 102801. 陈洋, 范荣双, 王竞雪, 等. 结合最小噪声分离变换和卷积神经网络的高分辨影像分类方法[J]. 激光与光电子学进展, 2017, 54(10): 102801.
[11] Penatti O A B, Nogueira K, dos Santos J A. Do deep features generalize from everyday objects to remote sensing and aerial scenes domains?[C]. IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2015: 44-51.
[12] Hu F, Xia G S, Hu J W, et al. Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery[J]. Remote Sensing, 2015, 7(11): 14680-14707.
[13] Yandex A B, Lempitsky V. Aggregating local deep features for image retrieval[C]. IEEE International Conference on Computer Vision, 2015: 1269-1277.
[14] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition [EB/OL]. (2015-04-10)[2018-09-28].https://arxiv.org/abs/1409.1556
[15] Yang C, Lu X, Lin Z, et al. High-resolution image inpainting using multi-scale neural patch synthesis[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4076-4084.
[16] Gong Y C, Wang L W, Guo R Q, et al. Multi-scale orderless pooling of deep convolutional activation features[C]. European Conference on Computer Vision, 2014:392-407.
[17] Zhao B, Zhong Y F, Xia G S, et al. Dirichlet-derived multiple topic scene classification model for high spatial resolution remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(4): 2108-2123.
[18] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks [C]. International Conference on Neural Information Processing Systems, 2012: 1097-1105.
[19] Jia Y, Shelhamer E, Donahue J, et al. Caffe: Convolutional architecture for fast feature embedding[C]. The ACM International Conference on Multimedia, 2014: 675-678.
[20] Chatfield K, Simonyan K, Vedaldi A, et al. Return of the devil in the details: delving deep into convolutional nets[EB/OL]. (2014-11-05)[2018-09-28]. https://arxiv.org/abs/1405.3531
[21] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
[2] Csurka G. Visual categorization with bags of keypoints[C]. European Conference on Computer Vision, 2004: 1-22.
[3] Lazebnik S, Schmid C, Ponce J. Beyond bags of features: spatial pyramid matching for recognizing natural scene categories[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2006: 2169-2178.
[4] Yang Y, Newsam S. Spatial pyramid co-occurrence for image classification[C]. International Conference on Computer Vision, 2011: 1465-1472.
[5] Fang X, Wang G H, Yang H C, et al. High resolution remote sensing image classification combining with mean-shift segmentation and fully convolution neural network[J]. Laser & Optoelectronics Progress, 2018, 55(2): 022802. 方旭, 王光辉, 杨化超, 等. 结合均值漂移分割与全卷积神经网络的高分辨遥感影像分类[J]. 激光与光电子学进展, 2018, 55(2): 022802.
[6] Liu D W, Han L, Han X Y. High spatial resolution remote sensing image classification based on deep learning[J]. Acta Optica Sinica, 2016, 36(4): 0428001. 刘大伟, 韩玲, 韩晓勇. 基于深度学习的高分辨率遥感影像分类研究[J]. 光学学报, 2016, 36(4): 0428001.
[7] Liu F, Lu L X, Huang G W, et al. Landform image classification based on discrete cosine transformation and deep network [J]. Acta Optica Sinica, 2018, 38(6): 0620001. 刘芳, 路丽霞, 黄光伟, 等. 基于离散余弦变换和深度网络的地貌图像分类[J]. 光学学报, 2018, 38(6): 0620001.
[8] Castelluccio M, Poggi G, Sansone C, et al. Land use classification in remote sensing images by convolutional neural networks[J]. Acta Ecologica Sinica, 2015, 28(2): 627-635.
[9] Zhong Y F, Fei F, Zhang L P. Large patch convolutional neural networks for the scene classification of high spatial resolution imagery[J]. Journal of Applied Remote Sensing, 2016, 10(2): 025006.
[10] Chen Y, Fan R S, Wang J X, et al. High resolution image classification method combining with minimum noise fraction rotation and convolution neural network[J]. Laser & Optoelectronics Progress, 2017, 54(10): 102801. 陈洋, 范荣双, 王竞雪, 等. 结合最小噪声分离变换和卷积神经网络的高分辨影像分类方法[J]. 激光与光电子学进展, 2017, 54(10): 102801.
[11] Penatti O A B, Nogueira K, dos Santos J A. Do deep features generalize from everyday objects to remote sensing and aerial scenes domains?[C]. IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2015: 44-51.
[12] Hu F, Xia G S, Hu J W, et al. Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery[J]. Remote Sensing, 2015, 7(11): 14680-14707.
[13] Yandex A B, Lempitsky V. Aggregating local deep features for image retrieval[C]. IEEE International Conference on Computer Vision, 2015: 1269-1277.
[14] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition [EB/OL]. (2015-04-10)[2018-09-28].https://arxiv.org/abs/1409.1556
[15] Yang C, Lu X, Lin Z, et al. High-resolution image inpainting using multi-scale neural patch synthesis[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4076-4084.
[16] Gong Y C, Wang L W, Guo R Q, et al. Multi-scale orderless pooling of deep convolutional activation features[C]. European Conference on Computer Vision, 2014:392-407.
[17] Zhao B, Zhong Y F, Xia G S, et al. Dirichlet-derived multiple topic scene classification model for high spatial resolution remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(4): 2108-2123.
[18] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks [C]. International Conference on Neural Information Processing Systems, 2012: 1097-1105.
[19] Jia Y, Shelhamer E, Donahue J, et al. Caffe: Convolutional architecture for fast feature embedding[C]. The ACM International Conference on Multimedia, 2014: 675-678.
[20] Chatfield K, Simonyan K, Vedaldi A, et al. Return of the devil in the details: delving deep into convolutional nets[EB/OL]. (2014-11-05)[2018-09-28]. https://arxiv.org/abs/1405.3531
[21] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.