小样本的多模态遥感影像高层特征融合分类
|
摘要
在使用深度学习模型研究遥感影像地物分类问题时,某些地物的遥感影像可用于训练的样本很少。同时,多样化的遥感影像获取方式产生了大量不同空间分辨率的多模态遥感影像。融合这些多模态遥感影像,弥补样本量少导致分类精度低的缺陷,是小样本的遥感影像高精度分类领域中亟待解决的问题。针对上述问题,提出了考虑两种空间分辨率遥感影像相关关系的融合分类方法。首先,使用两个并行的深度学习网络分别提取两种空间分辨率影像的高层特征;其次,将提取到的高层特征通过融合方法进行融合;最后,得到融合后的高层特征作为输入,训练整个融合分类模型。实验表明,不同融合策略的分类精度不同,本文提出的基于高层特征级别的融合策略可以有效提高分类精度。
The training sample size for some objects on the ground is quite small when applying a deep learning model to study the classification of remote sensing images.Meanwhile,diversified remote sensing image acquisition methods generate numerous multimodal remote sensing images with different spatial resolutions.Fusing these multi-modal remote sensing images to remedy the small sample size defect and achieve a highly precise classification of remote sensing images is an urgent problem to be solved.To this end,the present study proposes a fusion method for image classification based on the correlation of two spatial resolutions.A deep learning network is utilized to extract the high-level features of the remote sensing images in two spatial resolutions.Two types of high-level features are integrated via the proposed fusion strategy and further used as the input to train the whole network model.The experimental results demonstrate that the proposed fusion algorithm can achieve high classification accuracy.Further,because different fusion rules have different classification accuracies,a suitable selection can improve the classification accuracy.
The training sample size for some objects on the ground is quite small when applying a deep learning model to study the classification of remote sensing images.Meanwhile,diversified remote sensing image acquisition methods generate numerous multimodal remote sensing images with different spatial resolutions.Fusing these multi-modal remote sensing images to remedy the small sample size defect and achieve a highly precise classification of remote sensing images is an urgent problem to be solved.To this end,the present study proposes a fusion method for image classification based on the correlation of two spatial resolutions.A deep learning network is utilized to extract the high-level features of the remote sensing images in two spatial resolutions.Two types of high-level features are integrated via the proposed fusion strategy and further used as the input to train the whole network model.The experimental results demonstrate that the proposed fusion algorithm can achieve high classification accuracy.Further,because different fusion rules have different classification accuracies,a suitable selection can improve the classification accuracy.
引文
[1]Tanha J,van Someren M,Afsarmanesh H.Semisupervised self-training for decision tree classifiers[J].International Journal of Machine Learning and Cybernetics,2017,8(1):355-370.
[2]Guo Y,Jia X,Paull D.Sequential classifier training for rice mapping with multitemporal remote sensing imagery[J].ISPRS Annals of Photogrammetry,Remote Sensing and Spatial Information Sciences,2017,IV-4/W2:161-165.
[3]Xia M,Cao G,Wang G Y,et al.Remote sensing image classification based on deep learning and conditional random fields[J].Journal of Image and Graphics,2017,22(9):1289-1301.夏梦,曹国,汪光亚,等.结合深度学习与条件随机场的遥感图像分类[J].中国图象图形学报,2017,22(9):1289-1301.
[4]Cao X Y,Xu L,Meng D Y,et al.Integration of 3-dimensional discrete wavelet transform and Markov random field for hyperspectral image classification[J].Neurocomputing,2017,226:90-100.
[5]Krizhevsky A,Sutskever I,Hinton G E.ImageNet classification with deep convolutional neural networks[J].Communications of the ACM,2017,60(6):84-90.
[6]Liu Y T,Li Z Q,Yang X L.Application of improved convolution neural network in remote sensing image classification[J].Journal of Computer Applications,2018,38(4):949-954.刘雨桐,李志清,杨晓玲.改进卷积神经网络在遥感图像分类中的应用[J].计算机应用,2018,38(4):949-954.
[7]Yang H,Li L Q,Yang H H,et al.Method of urban management cases'image classification based on convolutional neural network[J].Computer Engineering and Applications,2018,54(10):242-248,266.杨浩,李灵巧,杨辉华,等.基于卷积神经网络的城管案件图像分类方法[J].计算机工程与应用,2018,54(10):242-248,266.
[8]Fu G Y,Gu H Y,Wang H Q.Spectral and spatial classification of hyperspectral images based on convolutional neural networks[J].Science Technology and Engineering,2017,17(21):268-274.付光远,辜弘炀,汪洪桥.基于卷积神经网络的高光谱图像谱-空联合分类[J].科学技术与工程,2017,17(21):268-274.
[9]LeCun Y,Bottou L,Bengio Y,et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[10]Simonyan K,Zisserman A.Very deep convolutional networks for large-scale image recognition[EB/OL].(2015-04-10)[2018-10-20].https:∥arxiv.org/abs/1409.1556.
[11]Eitel A,Springenberg J T,Spinello L,et al.Multimodal deep learning for robust RGB-D object recognition[C]∥2015 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS),September 28-October 2,2015,Hamburg,Germany.New York:IEEE,2015:15666832.
[12]Xu Y,Du J,Dai L R,et al.Cross-language transfer learning for deep neural network based speech enhancement[C]∥The 9th International Symposium on Chinese Spoken Language Processing,September12-14,2014,Singapore,Singapore.New York:IEEE,2014:14700579.
[13]Han D M,Liu Q G,Fan W G.A new image classification method using CNN transfer learning and web data augmentation[J].Expert Systems With Applications,2018,95:43-56.
[14]Ahmed A,Yu K,Xu W,et al.Training hierarchical feed-forward visual recognition models using transfer learning from pseudo-tasks[M]∥Forsyth D,Torr P,Zisserman A.Lecture Notes in Computer Science.Berlin,Heidelberg:Springer,2008:69-82.
[15]Camps-Valls G,Gomez-Chova L,Munoz-Mari J,et al.Kernel-based framework for multitemporal and multisource remote sensing data classification and change detection[J].IEEE Transactions on Geoscience and Remote Sensing、,2008,46(6):1822-1835.
[16]Ding X,Jiang Y,Huang Y,et al.Pan-sharpening with a Bayesian nonparametric dictionary learning model[C]∥Artificial Intelligence and Statistics.[S.l.]:[s.n.],2014:176-184.
[17]Hane C,Zach C,Cohen A,et al.Joint 3Dscene reconstruction and class segmentation[C]∥2013IEEE Conference on Computer Vision and Pattern Recognition,June 23-28,2013,Portland,OR,USA.New York:IEEE,2013:13824292.
[18]Fang X,Wang G H,Yang H C,et al.High resolution remote sensing image classification combining with mean-shift segmentation and fully convolution neural network[J].Laser&Optoelectronics Progress,2018,55(2):022802.方旭,王光辉,杨化超,等.结合均值漂移分割与全卷积神经网络的高分辨遥感影像分类[J].激光与光电子学进展,2018,55(2):022802.
[19]Tu S Q,Xue Y J,Liang Y,et al.Review on RGB-Dimage classification[J].Laser&Optoelectronics Progress,2016,53(6):060003.涂淑琴,薛月菊,梁云,等.RGB-D图像分类方法研究综述[J].激光与光电子学进展,2016,53(6):060003.
[20]He M Y,Cheng Y L,Liao X J,et al.Building extraction algorithm by fusing spectral and geometrical features[J].Laser&Optoelectronics Progress,2018,55(4):042803.何曼芸,程英蕾,廖湘江,等.融合光谱特征和几何特征的建筑物提取算法[J].激光与光电子学进展,2018,55(4):042803.
[21]Couprie C,Farabet C,Najman L,et al.Indoor semantic segmentation using depth information[EB/OL].(2013-03-14)[2018-10-20].https:∥arxiv.org/abs/1301.3572.
[22]Wang W,Yang X Y,Ooi B C,et al.Effective deep learning-based multi-modal retrieval[J].The VLDBJournal,2016,25(1):79-101.
[23]Cheng G,Han J W.A survey on object detection in optical remote sensing images[J].ISPRS Journal of Photogrammetry and Remote Sensing,2016,117:11-28.
[24]Cheng G,Zhou P C,Han J W.Learning rotationinvariant convolutional neural networks for object detection in VHR optical remote sensing images[J].IEEE Transactions on Geoscience and Remote Sensing,2016,54(12):7405-7415.
[25]Zhou R T,Li Z P,Wu C,et al.Buddy routing:a routing paradigm for NanoNets based on physical layer network coding[C]∥2012 21st International Conference on Computer Communications and Networks(ICCCN),July 30-August 2,2012,Munich,Germany.New York:IEEE,2012:12965366.
[2]Guo Y,Jia X,Paull D.Sequential classifier training for rice mapping with multitemporal remote sensing imagery[J].ISPRS Annals of Photogrammetry,Remote Sensing and Spatial Information Sciences,2017,IV-4/W2:161-165.
[3]Xia M,Cao G,Wang G Y,et al.Remote sensing image classification based on deep learning and conditional random fields[J].Journal of Image and Graphics,2017,22(9):1289-1301.夏梦,曹国,汪光亚,等.结合深度学习与条件随机场的遥感图像分类[J].中国图象图形学报,2017,22(9):1289-1301.
[4]Cao X Y,Xu L,Meng D Y,et al.Integration of 3-dimensional discrete wavelet transform and Markov random field for hyperspectral image classification[J].Neurocomputing,2017,226:90-100.
[5]Krizhevsky A,Sutskever I,Hinton G E.ImageNet classification with deep convolutional neural networks[J].Communications of the ACM,2017,60(6):84-90.
[6]Liu Y T,Li Z Q,Yang X L.Application of improved convolution neural network in remote sensing image classification[J].Journal of Computer Applications,2018,38(4):949-954.刘雨桐,李志清,杨晓玲.改进卷积神经网络在遥感图像分类中的应用[J].计算机应用,2018,38(4):949-954.
[7]Yang H,Li L Q,Yang H H,et al.Method of urban management cases'image classification based on convolutional neural network[J].Computer Engineering and Applications,2018,54(10):242-248,266.杨浩,李灵巧,杨辉华,等.基于卷积神经网络的城管案件图像分类方法[J].计算机工程与应用,2018,54(10):242-248,266.
[8]Fu G Y,Gu H Y,Wang H Q.Spectral and spatial classification of hyperspectral images based on convolutional neural networks[J].Science Technology and Engineering,2017,17(21):268-274.付光远,辜弘炀,汪洪桥.基于卷积神经网络的高光谱图像谱-空联合分类[J].科学技术与工程,2017,17(21):268-274.
[9]LeCun Y,Bottou L,Bengio Y,et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[10]Simonyan K,Zisserman A.Very deep convolutional networks for large-scale image recognition[EB/OL].(2015-04-10)[2018-10-20].https:∥arxiv.org/abs/1409.1556.
[11]Eitel A,Springenberg J T,Spinello L,et al.Multimodal deep learning for robust RGB-D object recognition[C]∥2015 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS),September 28-October 2,2015,Hamburg,Germany.New York:IEEE,2015:15666832.
[12]Xu Y,Du J,Dai L R,et al.Cross-language transfer learning for deep neural network based speech enhancement[C]∥The 9th International Symposium on Chinese Spoken Language Processing,September12-14,2014,Singapore,Singapore.New York:IEEE,2014:14700579.
[13]Han D M,Liu Q G,Fan W G.A new image classification method using CNN transfer learning and web data augmentation[J].Expert Systems With Applications,2018,95:43-56.
[14]Ahmed A,Yu K,Xu W,et al.Training hierarchical feed-forward visual recognition models using transfer learning from pseudo-tasks[M]∥Forsyth D,Torr P,Zisserman A.Lecture Notes in Computer Science.Berlin,Heidelberg:Springer,2008:69-82.
[15]Camps-Valls G,Gomez-Chova L,Munoz-Mari J,et al.Kernel-based framework for multitemporal and multisource remote sensing data classification and change detection[J].IEEE Transactions on Geoscience and Remote Sensing、,2008,46(6):1822-1835.
[16]Ding X,Jiang Y,Huang Y,et al.Pan-sharpening with a Bayesian nonparametric dictionary learning model[C]∥Artificial Intelligence and Statistics.[S.l.]:[s.n.],2014:176-184.
[17]Hane C,Zach C,Cohen A,et al.Joint 3Dscene reconstruction and class segmentation[C]∥2013IEEE Conference on Computer Vision and Pattern Recognition,June 23-28,2013,Portland,OR,USA.New York:IEEE,2013:13824292.
[18]Fang X,Wang G H,Yang H C,et al.High resolution remote sensing image classification combining with mean-shift segmentation and fully convolution neural network[J].Laser&Optoelectronics Progress,2018,55(2):022802.方旭,王光辉,杨化超,等.结合均值漂移分割与全卷积神经网络的高分辨遥感影像分类[J].激光与光电子学进展,2018,55(2):022802.
[19]Tu S Q,Xue Y J,Liang Y,et al.Review on RGB-Dimage classification[J].Laser&Optoelectronics Progress,2016,53(6):060003.涂淑琴,薛月菊,梁云,等.RGB-D图像分类方法研究综述[J].激光与光电子学进展,2016,53(6):060003.
[20]He M Y,Cheng Y L,Liao X J,et al.Building extraction algorithm by fusing spectral and geometrical features[J].Laser&Optoelectronics Progress,2018,55(4):042803.何曼芸,程英蕾,廖湘江,等.融合光谱特征和几何特征的建筑物提取算法[J].激光与光电子学进展,2018,55(4):042803.
[21]Couprie C,Farabet C,Najman L,et al.Indoor semantic segmentation using depth information[EB/OL].(2013-03-14)[2018-10-20].https:∥arxiv.org/abs/1301.3572.
[22]Wang W,Yang X Y,Ooi B C,et al.Effective deep learning-based multi-modal retrieval[J].The VLDBJournal,2016,25(1):79-101.
[23]Cheng G,Han J W.A survey on object detection in optical remote sensing images[J].ISPRS Journal of Photogrammetry and Remote Sensing,2016,117:11-28.
[24]Cheng G,Zhou P C,Han J W.Learning rotationinvariant convolutional neural networks for object detection in VHR optical remote sensing images[J].IEEE Transactions on Geoscience and Remote Sensing,2016,54(12):7405-7415.
[25]Zhou R T,Li Z P,Wu C,et al.Buddy routing:a routing paradigm for NanoNets based on physical layer network coding[C]∥2012 21st International Conference on Computer Communications and Networks(ICCCN),July 30-August 2,2012,Munich,Germany.New York:IEEE,2012:12965366.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |