联合显著性特征与卷积神经网络的遥感影像舰船检测
|
摘要
目的针对高分辨率遥感影像舰船检测受云雾、海浪以及海岛等复杂因素干扰,存在虚警率高、漏检率高、目标检测和识别困难等问题,提出一种联合视觉显著性特征与卷积神经网络的海面舰船目标检测方法。方法基于频率域相位谱显著性检测能够有效抑制高分辨率遥感影像上云层、海面杂波干扰的特点,计算影像多尺度显著图并进行加权融合。采用对数变换对融合后的图像进行空间域灰度增强以提高目标与背景的区分度,利用灰度形态学闭运算填充舰船目标孔洞,采用大津分割法来提取疑似舰船目标作为兴趣区域。最后构建舰船样本库,利用迁移学习的思想训练卷积神经网络模型,对所有兴趣区域切片进行分类判断和识别,得到最终检测结果。结果利用多幅不同背景下的高分辨率遥感影像,分别从视觉显著性检测、舰船粗检测与船只类型识别3个方面进行实验验证,选取检测率、虚警率、识别率3个指标进行定量评价。结果表明,本文方法相比于其他方法能有效排除云雾、海岛等多种因素的干扰,检测率、虚警率、识别率分别为93. 63%、3. 01%、90. 09%,明显优于其他算法,能够实现大范围影像上多种类型舰船的快速准确检测和识别。结论本文将图像视觉显著性检测快速获取图像显著目标的特点与卷积神经网络在图像分类的优势相结合,应用于遥感影像的海域舰船目标检测,能够实现对复杂背景下舰船目标的检测和船只类型的精细化识别。
Objective Ships and warships are important sea-based transportation carriers and military targets. Thus,detecting and recognizing these targets in high resolution remote sensing images are of substantial practical significance to. However,satellite imaging can be affected by weather,illumination,cloud,and atmosphere scattering. In addition,the targets in the image can be disturbed by sea clutters and other objects,which render the ship detection and recognition increasingly difficult. The majority of ship recognition algorithms typically adopt low-level features,such as shape,invariant moment,and histogram of gradient( HOG),which are simple but not robust to disturbances,such as waves,clouds,and islands.In general,handcraft features can only be used to distinguish ships from other interferences on the sea surface and have weak ability to differentiate various types of ship. In view of the above mentioned problems,this study proposes a new method that combines salient features and a convolutional neural network to recognize ships in remote sensing images. MethodThe proposed method consists of three parts,namely,image pre-processing,ship pre-detection,and ship recognition.First,the image can be enhanced by a homomorphic filter to improve the texture clarity and contrast in the pre-processing phase,which is helpful for the detection and recognition of the subsequent phase. In the ship detection stage,the saliency map of images can be calculated by phase spectrum of Fourier transform( PFT),which is a technique based on the analysis of the frequency domain. To take account of different resolutions,the multi-scale saliency maps are fused. The PFT method can effectively suppress the interference of cloud and sea wave,but the distinction between background and ship is barely notable. To solve this problem,logarithmic transformation is utilized to enhance the saliency map. Then,the gray morphological operation of close is adapted to eliminate the noise areas and fill holes,and the image segmentation algorithm of Otsu is used to extract all salient areas as areas of interest. In the stage of recognition,a deep convolutional neural network( CNN) can be well trained with a small number of ship samples based on the concept of transfer learning. All areas of interest can be finally classified and recognized by the CNN. Result To verify the effectiveness of the proposed algorithm,experiments were conducted on remote sensing images with varying backgrounds. The experiments were conducted in three aspects,namely,visual saliency detection,ship detection,and ship recognition. Three kinds of indicators,namely,detection,false alarm,and recognition rates were used to quantify experimental results. The qualitative results indicate that saliency detection based on PFT can effectively restrain the disturbance of sea surface and clutter,in which logarithmic transformation substantially improves the integrity of the ships' contour. Quantitative analysis shows that the three indexes of the proposed method are 93. 63%,3. 01%,and 90. 09%,respectively,which are extensively better than the compared algorithms. Conclusion Visual saliency detection is one of the commonly used and effective methods for ship detection. This paper combined the advantages of visual salient features and convolutional neural network for ship recognition in remote sensing images. The method can realize the rapid detection of ship targets with high accuracy of classification in complex backgrounds.
Objective Ships and warships are important sea-based transportation carriers and military targets. Thus,detecting and recognizing these targets in high resolution remote sensing images are of substantial practical significance to. However,satellite imaging can be affected by weather,illumination,cloud,and atmosphere scattering. In addition,the targets in the image can be disturbed by sea clutters and other objects,which render the ship detection and recognition increasingly difficult. The majority of ship recognition algorithms typically adopt low-level features,such as shape,invariant moment,and histogram of gradient( HOG),which are simple but not robust to disturbances,such as waves,clouds,and islands.In general,handcraft features can only be used to distinguish ships from other interferences on the sea surface and have weak ability to differentiate various types of ship. In view of the above mentioned problems,this study proposes a new method that combines salient features and a convolutional neural network to recognize ships in remote sensing images. MethodThe proposed method consists of three parts,namely,image pre-processing,ship pre-detection,and ship recognition.First,the image can be enhanced by a homomorphic filter to improve the texture clarity and contrast in the pre-processing phase,which is helpful for the detection and recognition of the subsequent phase. In the ship detection stage,the saliency map of images can be calculated by phase spectrum of Fourier transform( PFT),which is a technique based on the analysis of the frequency domain. To take account of different resolutions,the multi-scale saliency maps are fused. The PFT method can effectively suppress the interference of cloud and sea wave,but the distinction between background and ship is barely notable. To solve this problem,logarithmic transformation is utilized to enhance the saliency map. Then,the gray morphological operation of close is adapted to eliminate the noise areas and fill holes,and the image segmentation algorithm of Otsu is used to extract all salient areas as areas of interest. In the stage of recognition,a deep convolutional neural network( CNN) can be well trained with a small number of ship samples based on the concept of transfer learning. All areas of interest can be finally classified and recognized by the CNN. Result To verify the effectiveness of the proposed algorithm,experiments were conducted on remote sensing images with varying backgrounds. The experiments were conducted in three aspects,namely,visual saliency detection,ship detection,and ship recognition. Three kinds of indicators,namely,detection,false alarm,and recognition rates were used to quantify experimental results. The qualitative results indicate that saliency detection based on PFT can effectively restrain the disturbance of sea surface and clutter,in which logarithmic transformation substantially improves the integrity of the ships' contour. Quantitative analysis shows that the three indexes of the proposed method are 93. 63%,3. 01%,and 90. 09%,respectively,which are extensively better than the compared algorithms. Conclusion Visual saliency detection is one of the commonly used and effective methods for ship detection. This paper combined the advantages of visual salient features and convolutional neural network for ship recognition in remote sensing images. The method can realize the rapid detection of ship targets with high accuracy of classification in complex backgrounds.
引文
[1]Wang Y Q,Ma L,Tian Y.State-of-the-art of ship detection and recognition in optical remotely sensed imagery[J].Acta Automatica Sinica,2011,37(9):1029-1039.[王彦情,马雷,田原.光学遥感图像舰船目标检测与识别综述[J].自动化学报,2011,37(9):1029-1039.][DOI:10.3724/SP.J.1004.2011.01029]
[2]Tang M E,Lin T Q,Wen G J.Overview of ship detection methods in remote sensing image[J].Application Research of Computers,2011,28(1):29-36.[唐沐恩,林挺强,文贡坚.遥感图像中舰船检测方法综述[J].计算机应用研究,2011,28(1):29-36.][DOI:10.3969/j.issn.1001-3695.2011.01.007]
[3]Zhou J,Tian J W.Method of detecting small target in port-sea background[J].Infrared and Laser Engineering,2005,34(4):486-489.[周静,田金文.基于海岸线背景的海面小目标检测方法[J].红外与激光工程,2005,34(4):486-489.][DOI:10.3969/j.issn.1007-2276.2005.04.026]
[4]Sun Y J,Lei W H,Hu Y H,et al.Ship target detection method based on multi-source remote sensing satellite[J].Laser&Infrared,2018,48(2):267-272.[孙越娇,雷武虎,胡以华,等.基于多源遥感卫星的海面舰船目标检测方法[J].激光与红外,2018,48(2):267-272.][DOI:10.3969/j.issn.1001-5078.2018.02.023]
[5]Wang B Y,Zhang R,Yuan Y,et al.A new multi-level threshold segmentation method for ship targets detection in optical remote sensing images[J].Journal of University of Science and Technology of China,2011,41(4):293-298.[王保云,张荣,袁圆,等.可见光遥感图像中舰船目标检测的多阶阈值分割方法[J].中国科学技术大学学报,2011,41(4):293-298.][DOI:10.3969/j.issn.0253-2778.2011.04.002]
[6]Wang F C,Zhang M,Gong L M,et al.Fast detection algorithm for ships under the background of ocean[J].Laser&Infrared,2016,46(5):602-606.[王方超,张旻,宫丽美,等.海洋背景下的海面舰船快速检测算法[J].激光与红外,2016,46(5):602-606.][DOI:10.3969/j.issn.1001-5078.2016.05.018]
[7]Zhang X D,He S H,Yang S Q.Ship targets detection method based on multi-scale fractal feature[J].Laser&Infrared,2009,39(3):315-318.[张东晓,何四华,杨绍清.一种多尺度分形的舰船目标检测方法[J].激光与红外,2009,39(3):315-318.][DOI:10.3969/j.issn.1001-5078.2009.03.023]
[8]Ding P,Zhang Y,Jia P,et al.Ship detection on sea surface based on multi-feature and multi-scale visual attention[J].Optics and Precision Engineering,2017,25(9):2461-2468.[丁鹏,张叶,贾平,等.基于多尺度多特征视觉显著性的海面舰船检测[J].光学精密工程,2017,25(9):2461-2468.][DOI:10.3788/ope.20172509.2461]
[9]Cheng H,Liu S T,Sun W B,et al.Fine rapid detection of ship targets in remote sensing images[J].Opto-Electronic Engineering,2016,43(4):25-32.[程红,刘思彤,孙文邦,等.遥感图像中舰船目标的快速精细检测[J].光电工程,2016,43(4):25-32.][DOI:10.3969/j.issn.1003-501X.2016.04.005]
[10]Xu F,Liu J H,Zeng D D,et al.Detection and identification of unsupervised ships and warships on sea surface based on visual saliency[J].Optics and Precision Engineering,2017,25(5):1300-1311.[徐芳,刘晶红,曾冬冬,等.基于视觉显著性的无监督海面舰船检测与识别[J].光学精密工程,2017,25(5):1300-1311.][DOI:10.3788/OPE.20172505.1300]
[11]Wang H L.Research on detection of typical man-made objects in high resolution optical remote sensing images[D].Changchun:Changchun Institute of Optics,Fine Mehcanics and Physics,Chinese Academy of Sciences,2017.[王慧利.高分辨率光学遥感图像中典型人造目标检测技术研究[D].长春:中国科学院长春光学精密机械与物理研究所,2017.]
[12]Hou X D,Zhang L Q.Saliency detection:a spectral residual approach[C]//Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition.Minneapolis,MN,USA:IEEE,2007:1-8.[DOI:10.1109/CVPR.2007.383267]
[13]Chen H Y,Xu S,Liu K,et al.Surface defect detection of steel strip based on spectral residual visual saliency[J].Optics and Precision Engineering,2016,24(10):2572-2580.[陈海永,徐森,刘坤,等.基于谱残差视觉显著性的带钢表面缺陷检测[J].光学精密工程,2016,24(10):2572-2580.][DOI:10.3788/OPE.20162410.2572]
[14]Guo C L,Ma Q,Zhang L M.Spatio-temporal Saliency detection using phase spectrum of quaternion Fourier transform[C]//Proceedings of 2008 IEEE Conference on Computer Vision and Pattern Recognition.Anchorage,AK,USA:IEEE,2008:1-8.[DOI:10.1109/CVPR.2008.4587715]
[15]Deng J,Dong W,Socher R,et al.Image Net:a large-scale hierarchical image database[C]//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition.Miami,FL,USA:IEEE,2009:248-255.[DOI:10.1109/CVPR.2009.5206848]
[16]Krizhevsky A.Learning multiple layers of features from tiny images[R].Toronto:Science Department,University of Toronto,2009.
[17]He K M,Zhang X Y,Ren S Q,et al.Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,NV,USA:IEEE,2016:770-778.[DOI:10.1109/CVPR.2016.90]
[18]Achanta R,Hemami S,Estrada F,et al.Frequency-tuned salient region detection[C]//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition.Miami,FL,USA:IEEE,2009:1597-1604.[DOI:10.1109/CVPR.2009.5206596]
[2]Tang M E,Lin T Q,Wen G J.Overview of ship detection methods in remote sensing image[J].Application Research of Computers,2011,28(1):29-36.[唐沐恩,林挺强,文贡坚.遥感图像中舰船检测方法综述[J].计算机应用研究,2011,28(1):29-36.][DOI:10.3969/j.issn.1001-3695.2011.01.007]
[3]Zhou J,Tian J W.Method of detecting small target in port-sea background[J].Infrared and Laser Engineering,2005,34(4):486-489.[周静,田金文.基于海岸线背景的海面小目标检测方法[J].红外与激光工程,2005,34(4):486-489.][DOI:10.3969/j.issn.1007-2276.2005.04.026]
[4]Sun Y J,Lei W H,Hu Y H,et al.Ship target detection method based on multi-source remote sensing satellite[J].Laser&Infrared,2018,48(2):267-272.[孙越娇,雷武虎,胡以华,等.基于多源遥感卫星的海面舰船目标检测方法[J].激光与红外,2018,48(2):267-272.][DOI:10.3969/j.issn.1001-5078.2018.02.023]
[5]Wang B Y,Zhang R,Yuan Y,et al.A new multi-level threshold segmentation method for ship targets detection in optical remote sensing images[J].Journal of University of Science and Technology of China,2011,41(4):293-298.[王保云,张荣,袁圆,等.可见光遥感图像中舰船目标检测的多阶阈值分割方法[J].中国科学技术大学学报,2011,41(4):293-298.][DOI:10.3969/j.issn.0253-2778.2011.04.002]
[6]Wang F C,Zhang M,Gong L M,et al.Fast detection algorithm for ships under the background of ocean[J].Laser&Infrared,2016,46(5):602-606.[王方超,张旻,宫丽美,等.海洋背景下的海面舰船快速检测算法[J].激光与红外,2016,46(5):602-606.][DOI:10.3969/j.issn.1001-5078.2016.05.018]
[7]Zhang X D,He S H,Yang S Q.Ship targets detection method based on multi-scale fractal feature[J].Laser&Infrared,2009,39(3):315-318.[张东晓,何四华,杨绍清.一种多尺度分形的舰船目标检测方法[J].激光与红外,2009,39(3):315-318.][DOI:10.3969/j.issn.1001-5078.2009.03.023]
[8]Ding P,Zhang Y,Jia P,et al.Ship detection on sea surface based on multi-feature and multi-scale visual attention[J].Optics and Precision Engineering,2017,25(9):2461-2468.[丁鹏,张叶,贾平,等.基于多尺度多特征视觉显著性的海面舰船检测[J].光学精密工程,2017,25(9):2461-2468.][DOI:10.3788/ope.20172509.2461]
[9]Cheng H,Liu S T,Sun W B,et al.Fine rapid detection of ship targets in remote sensing images[J].Opto-Electronic Engineering,2016,43(4):25-32.[程红,刘思彤,孙文邦,等.遥感图像中舰船目标的快速精细检测[J].光电工程,2016,43(4):25-32.][DOI:10.3969/j.issn.1003-501X.2016.04.005]
[10]Xu F,Liu J H,Zeng D D,et al.Detection and identification of unsupervised ships and warships on sea surface based on visual saliency[J].Optics and Precision Engineering,2017,25(5):1300-1311.[徐芳,刘晶红,曾冬冬,等.基于视觉显著性的无监督海面舰船检测与识别[J].光学精密工程,2017,25(5):1300-1311.][DOI:10.3788/OPE.20172505.1300]
[11]Wang H L.Research on detection of typical man-made objects in high resolution optical remote sensing images[D].Changchun:Changchun Institute of Optics,Fine Mehcanics and Physics,Chinese Academy of Sciences,2017.[王慧利.高分辨率光学遥感图像中典型人造目标检测技术研究[D].长春:中国科学院长春光学精密机械与物理研究所,2017.]
[12]Hou X D,Zhang L Q.Saliency detection:a spectral residual approach[C]//Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition.Minneapolis,MN,USA:IEEE,2007:1-8.[DOI:10.1109/CVPR.2007.383267]
[13]Chen H Y,Xu S,Liu K,et al.Surface defect detection of steel strip based on spectral residual visual saliency[J].Optics and Precision Engineering,2016,24(10):2572-2580.[陈海永,徐森,刘坤,等.基于谱残差视觉显著性的带钢表面缺陷检测[J].光学精密工程,2016,24(10):2572-2580.][DOI:10.3788/OPE.20162410.2572]
[14]Guo C L,Ma Q,Zhang L M.Spatio-temporal Saliency detection using phase spectrum of quaternion Fourier transform[C]//Proceedings of 2008 IEEE Conference on Computer Vision and Pattern Recognition.Anchorage,AK,USA:IEEE,2008:1-8.[DOI:10.1109/CVPR.2008.4587715]
[15]Deng J,Dong W,Socher R,et al.Image Net:a large-scale hierarchical image database[C]//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition.Miami,FL,USA:IEEE,2009:248-255.[DOI:10.1109/CVPR.2009.5206848]
[16]Krizhevsky A.Learning multiple layers of features from tiny images[R].Toronto:Science Department,University of Toronto,2009.
[17]He K M,Zhang X Y,Ren S Q,et al.Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,NV,USA:IEEE,2016:770-778.[DOI:10.1109/CVPR.2016.90]
[18]Achanta R,Hemami S,Estrada F,et al.Frequency-tuned salient region detection[C]//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition.Miami,FL,USA:IEEE,2009:1597-1604.[DOI:10.1109/CVPR.2009.5206596]