基于R-FCN的航拍巡检图像目标检测方法
|
摘要
航拍巡检是输电线路巡检的主要方式之一,目前的航拍巡检方式效率较低,受巡检员主观因素影响大,亟需一种智能检测算法自动定位并识别输电线路巡检图片中的故障。基于深度学习的航拍巡检图像目标检测技术作为一种可能的解决方案,得到了广泛关注。提出了一种利用基于区域的全卷积网络(R-FCN)的航拍巡检图像目标检测方法,并利用在线困难样本挖掘(OHEM)、样本优化、软性非极大值抑制(Soft-NMS)等改进方法进行优化。实验证明,所提方法具有目标定位准确、平均准确率高、单模型可同时检测目标种类多等特点。
Aerial inspection is one of the main methods of transmission line inspection. In consideration of inefficient of aerial inspection mode and subjective factors of inspectors, there is an urgent need for intelligent detection algorithm to locate and identify the faults in inspection pictures of transmission line. As a possible solution, object detection technology of aerial inspection imagge based on deep learning has attracted extensive attention. An object detection method of aerial inspection image utilizing region-based fully convolutional network(R-FCN) is proposed. Online hard example mining(OHEM), sample adjusting and soft non-maximum suppression(Soft-NMS) are adopted to improve the performance of the proposed algorithm. The experiment results show that the proposed method has obvious advantages on accurate target location, high average precision, and simultaneous detection of target species by single-model.
Aerial inspection is one of the main methods of transmission line inspection. In consideration of inefficient of aerial inspection mode and subjective factors of inspectors, there is an urgent need for intelligent detection algorithm to locate and identify the faults in inspection pictures of transmission line. As a possible solution, object detection technology of aerial inspection imagge based on deep learning has attracted extensive attention. An object detection method of aerial inspection image utilizing region-based fully convolutional network(R-FCN) is proposed. Online hard example mining(OHEM), sample adjusting and soft non-maximum suppression(Soft-NMS) are adopted to improve the performance of the proposed algorithm. The experiment results show that the proposed method has obvious advantages on accurate target location, high average precision, and simultaneous detection of target species by single-model.
引文
[1] 国家能源局,中国电力企业联合会.2017年全国电力可靠性年度报告[R].北京:国家能源局,2018.National Energy Administration,China Electricity Council.2017 annual report of national power reliability[R].Beijing:National Energy Administration,2018.
[2] 仝卫国,苑津莎,李宝树.图像处理技术在直升机巡检输电线路中的应用综述[J].电网技术,2010,34(12):204-208.TONG Weiguo,YUAN Jinsha,LI Baoshu.Application of image processing in patrol inspection of overhead transmission line by helicopter[J].Power System Technology,2010,34(12):204-208.
[3] 杨挺,赵黎媛,王成山.人工智能在电力系统及综合能源系统中的应用综述[J].电力系统自动化,2019,43(1):2-14.DOI:10.7500/AEPS20180706005.YANG Ting,ZHAO Liyuan,WANG Chengshan.Review on application of artificial intelligence in power system and integrated energy system[J].Automation of Electric Power Systems,2019,43(1):2-14.DOI:10.7500/AEPS 20180706005.
[4] 程乐峰,余涛,张孝顺,等.机器学习在能源与电力系统领域的应用和展望[J].电力系统自动化,2019,43(1):15-43.DOI:10.7500/AEPS20180814007.CHENG Lefeng,YU Tao,ZHANG Xiaoshun,et al.Machine learning for energy and electric power systems:state of the art and prospects[J].Automation of Electric Power Systems,2019,43(1):15-43.DOI:10.7500/AEPS20180814007.
[5] 赵振兵,金思新,刘亚春.基于NSCT的航拍绝缘子图像边缘提取方法[J].仪器仪表学报,2012,33(9):2045-2052.ZHAO Zhenbing,JIN Sixin,LIU Yachun.Aerial insulator image edge extraction method based on NSCT[J].Chinese Journal of Scientific Instrument,2012,33(9):2045-2052.
[6] 黄宵宁,张真良.直升机巡检航拍图像中绝缘子图像的提取算法[J].电网技术,2010,34(1):194-197.HUANG Xiaoning,ZHANG Zhenliang.A method to extract insulator image from aerial image of helicopter patrol[J].Power System Technology,2010,34(1):194-197.
[7] WU Q,AN J.An active contour model based on texture distribution for extracting inhomogeneous insulators from aerial images[J].IEEE Transactions on Geoscience and Remote Sensing,2014,52(6):3613-3626.
[8] 赵振兵,徐磊,戚银城,等.基于Hough检测和C-V模型的航拍绝缘子自动协同分割方法[J].仪器仪表学报,2016,37(2):395-403.ZHAO Zhenbing,XU Lei,QI Yincheng,et al.Automatic co-segmentation method for aerial insulator based on Hough detection and C-V model[J].Chinese Journal of Scientific Instrument,2016,37(2):395-403.
[9] 赵振兵,王乐.一种航拍绝缘子串图像自动定位方法[J].仪器仪表学报,2014,35(3):558-565.ZHAO Zhenbing,WANG Le.Aerial insulator string image automatic location method[J].Chinese Journal of Scientific Instrument,2014,35(3):558-565.
[10] 付晶,邵瑰玮,吴亮,等.利用层次模型进行训练学习的线路设备缺陷检测方法[J].高电压技术,2017,43(1):266-275.FU Jing,SHAO Guiwei,WU Liang,et al.Defect detection of line facility using hierarchical model with learning algorithm[J].High Voltage Engineering,2017,43(1):266-275.
[11] 金立军,胡娟,闫书佳.基于图像的高压输电线间隔棒故障诊断方法[J].高电压技术,2013,39(5):1040-1045.JIN Lijun,HU Juan,YAN Shujia.Method of spacer fault diagnose on transmission line based on image procession[J].High Voltage Engineering,2013,39(5):1040-1045.
[12] HOMMA R Z,SOHN O,BOSE R C.Analysis of the recognition and localisation techniques of power transmission lines components in aerial images acquired by drones[J].CIRED—Open Access Proceedings Journal,2017,2017(1):29-32.
[13] 黄新波,刘新慧,张烨,等.基于红蓝色差和改进K-means算法的航拍绝缘子分类识别方法[J].高电压技术,2018,44(5):1528-1534.HUANG Xinbo,LIU Xinhui,ZHANG Ye,et al.Classification recognition method of insulator in aerial image based on the red-blue difference and developed K-means algorithm[J].High Voltage Engineering,2018,44(5):1528-1534.
[14] ZHAI Y,CHEN R,YANG Q,et al.Insulator fault detection based on spatial morphological features of aerial images[J].IEEE Access,2018,6:35316-35326.
[15] 张烨,冯玲,穆靖宇,等.输电线路绝缘子覆冰厚度图像识别算法[J].电力系统自动化,2016,40(21):195-202.ZHANG Ye,FENG Ling,MU Jingyu,et al.Image identification algorithm of icing thickness for insulator in power transmission lines[J].Automation of Electric Power Systems,2016,40(21):195-202.
[16] 蒋兴良,夏云峰,张志劲,等.基于优化Gabor滤波器的输电导线断股图像检测[J].电力系统自动化,2011,35(15):78-83.JIANG Xingliang,XIA Yunfeng,ZHANG Zhijin,et al.Image detection for broken strand faults of transmission conductor based on optimized Gabor filter[J].Automation of Electric Power Systems,2011,35(15):78-83.
[17] 王万国,田兵,刘越,等.基于RCNN的无人机巡检图像电力小部件识别研究[J].地球信息科学学报,2017,19(2):256-263.WANG Wanguo,TIAN Bing,LIU Yue,et al.Study on the electrical devices detection in UAV images based on region based convolutional neural networks[J].Journal of Geo-information Science,2017,19(2):256-263.
[18] 陈庆,闫斌,叶润,等.航拍绝缘子卷积神经网络检测及自爆识别研究[J].电子测量与仪器学报,2017,31(6):942-953.CHEN Qing,YAN bin,YE Run,et al.Insulator detection and recognition of explosion fault based on convolutional neural networks[J].Journal of Electronic Measurement and Instrumentation,2017,31(6):942-953.
[19] 李军锋,王钦若,李敏.结合深度学习和随机森林的电力设备图像识别[J].高电压技术,2017,43(11):3705-3711.LI Junfeng,WANG Qinruo,LI Min.Electric equipment image recognition based on deep learning and random forest[J].High Voltage Engineering,2017,43(11):3705-3711.
[20] 汤踊,韩军,魏文力,等.深度学习在输电线路中部件识别与缺陷检测的研究[J].电子测量技术,2018,41(6):60-65.TANG Yong,HAN Jun,WEI Wenli,et al.Research on part recognition and defect detection of transmission line in deep learning[J].Electronic Measurement Technology,2018,41(6):60-65.
[21] 张骥,余娟,汪金礼,等.基于深度学习的输电线路外破图像识别技术[J].计算机系统应用,2018,27(8):176-179.ZHANG Ji,YU Juan,WANG Jinli,et al.Image recognition technology for transmission line external damage based on depth learning[J].Computer System & Applications,2018,27(8):176-179.
[22] DAI J,LI Y,HE K,et al.R-FCN:object detection via region-based fully convolutional networks[C]// Proceedings of the 30th International Conference on Neural Information Processing Systems,December 5-10,2016,Barcelona,Spain:379-387.
[23] REN S,GIRSHICK R,GIRSHICK R,et al.Faster R-CNN:towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2017,39(6):1137.
[24] REDMON J,FARHADI A.YOLOv3:an incremental improvement [EB/OL].[2018-08-01].https://arxiv.org/abs/1804.02767.
[25] SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition [EB/OL].[2018-08-01].https://arxiv.org/abs/1409.1556.
[26] SZEGEDY C,VANHOUCKE V,IOFFE S,et al.Rethinking the inception architecture for computer vision [EB/OL].[2018-08-01].https://arxiv.org/abs/1512.00567.
[27] HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition[EB/OL].[2018-08-01].https://arxiv.org/abs/1512.03385.
[28] XIE S,GIRSHICK R,DOLLAR P,et al.Aggregated residual transformations for deep neural networks [EB/OL].[2018-08-01].https://arxiv.org/abs/1611.05431.
[29] HU J,SHEN L,SUN G.Squeeze-and-excitation networks [EB/OL].[2018-08-01].https://arxiv.org/abs/1709.01507.
[30] REDMON J,DIVVALA S,GIRSHICK R,et al.You only look once:unified,real-time object detection [EB/OL].[2018-08-01].https://arxiv.org/abs/1506.02640.
[31] GIRSHICK R.Fast R-CNN [EB/OL].[2018-08-01].https://arxiv.org/abs/1504.08083.
[32] NEUBECK A,GOOL L V.Efficient non-maximum suppression[C]// 18th International Conference on Pattern Recognition,August 20-24,2006,Hong Kong,China:850-855.
[33] SHRIVASTAVA A,GUPTA A,GIRSHICK R.Training region-based object detectors with online hard example mining [EB/OL].[2018-08-01].https://arxiv.org/abs/1604.03540.
[34] BODLA N,SINGH B,CHELLAPPA R,et al.Soft-NMS:improving object detection with one line of code [EB/OL].[2018-08-01].https://arxiv.org/abs/1704.04503.
[35] LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection[C]// 2017 IEEE International Conference on Computer Vision,October 22-29,2017,Venice,Italy.
[36] CAI Z,VASCONCELOS N.Cascade R-CNN:delving into high quality object detection[EB/OL].[2018-08-01].https://arxiv.org/pdf/1712.00726.pdf.
[37] HE Y,ZHANG X,SAVVIDES M,et al.Softer-NMS:rethinking bounding box regression for accurate object detection[J].[EB/OL].[2018-08-01].https://arxiv.org/abs/1809.08545.
[2] 仝卫国,苑津莎,李宝树.图像处理技术在直升机巡检输电线路中的应用综述[J].电网技术,2010,34(12):204-208.TONG Weiguo,YUAN Jinsha,LI Baoshu.Application of image processing in patrol inspection of overhead transmission line by helicopter[J].Power System Technology,2010,34(12):204-208.
[3] 杨挺,赵黎媛,王成山.人工智能在电力系统及综合能源系统中的应用综述[J].电力系统自动化,2019,43(1):2-14.DOI:10.7500/AEPS20180706005.YANG Ting,ZHAO Liyuan,WANG Chengshan.Review on application of artificial intelligence in power system and integrated energy system[J].Automation of Electric Power Systems,2019,43(1):2-14.DOI:10.7500/AEPS 20180706005.
[4] 程乐峰,余涛,张孝顺,等.机器学习在能源与电力系统领域的应用和展望[J].电力系统自动化,2019,43(1):15-43.DOI:10.7500/AEPS20180814007.CHENG Lefeng,YU Tao,ZHANG Xiaoshun,et al.Machine learning for energy and electric power systems:state of the art and prospects[J].Automation of Electric Power Systems,2019,43(1):15-43.DOI:10.7500/AEPS20180814007.
[5] 赵振兵,金思新,刘亚春.基于NSCT的航拍绝缘子图像边缘提取方法[J].仪器仪表学报,2012,33(9):2045-2052.ZHAO Zhenbing,JIN Sixin,LIU Yachun.Aerial insulator image edge extraction method based on NSCT[J].Chinese Journal of Scientific Instrument,2012,33(9):2045-2052.
[6] 黄宵宁,张真良.直升机巡检航拍图像中绝缘子图像的提取算法[J].电网技术,2010,34(1):194-197.HUANG Xiaoning,ZHANG Zhenliang.A method to extract insulator image from aerial image of helicopter patrol[J].Power System Technology,2010,34(1):194-197.
[7] WU Q,AN J.An active contour model based on texture distribution for extracting inhomogeneous insulators from aerial images[J].IEEE Transactions on Geoscience and Remote Sensing,2014,52(6):3613-3626.
[8] 赵振兵,徐磊,戚银城,等.基于Hough检测和C-V模型的航拍绝缘子自动协同分割方法[J].仪器仪表学报,2016,37(2):395-403.ZHAO Zhenbing,XU Lei,QI Yincheng,et al.Automatic co-segmentation method for aerial insulator based on Hough detection and C-V model[J].Chinese Journal of Scientific Instrument,2016,37(2):395-403.
[9] 赵振兵,王乐.一种航拍绝缘子串图像自动定位方法[J].仪器仪表学报,2014,35(3):558-565.ZHAO Zhenbing,WANG Le.Aerial insulator string image automatic location method[J].Chinese Journal of Scientific Instrument,2014,35(3):558-565.
[10] 付晶,邵瑰玮,吴亮,等.利用层次模型进行训练学习的线路设备缺陷检测方法[J].高电压技术,2017,43(1):266-275.FU Jing,SHAO Guiwei,WU Liang,et al.Defect detection of line facility using hierarchical model with learning algorithm[J].High Voltage Engineering,2017,43(1):266-275.
[11] 金立军,胡娟,闫书佳.基于图像的高压输电线间隔棒故障诊断方法[J].高电压技术,2013,39(5):1040-1045.JIN Lijun,HU Juan,YAN Shujia.Method of spacer fault diagnose on transmission line based on image procession[J].High Voltage Engineering,2013,39(5):1040-1045.
[12] HOMMA R Z,SOHN O,BOSE R C.Analysis of the recognition and localisation techniques of power transmission lines components in aerial images acquired by drones[J].CIRED—Open Access Proceedings Journal,2017,2017(1):29-32.
[13] 黄新波,刘新慧,张烨,等.基于红蓝色差和改进K-means算法的航拍绝缘子分类识别方法[J].高电压技术,2018,44(5):1528-1534.HUANG Xinbo,LIU Xinhui,ZHANG Ye,et al.Classification recognition method of insulator in aerial image based on the red-blue difference and developed K-means algorithm[J].High Voltage Engineering,2018,44(5):1528-1534.
[14] ZHAI Y,CHEN R,YANG Q,et al.Insulator fault detection based on spatial morphological features of aerial images[J].IEEE Access,2018,6:35316-35326.
[15] 张烨,冯玲,穆靖宇,等.输电线路绝缘子覆冰厚度图像识别算法[J].电力系统自动化,2016,40(21):195-202.ZHANG Ye,FENG Ling,MU Jingyu,et al.Image identification algorithm of icing thickness for insulator in power transmission lines[J].Automation of Electric Power Systems,2016,40(21):195-202.
[16] 蒋兴良,夏云峰,张志劲,等.基于优化Gabor滤波器的输电导线断股图像检测[J].电力系统自动化,2011,35(15):78-83.JIANG Xingliang,XIA Yunfeng,ZHANG Zhijin,et al.Image detection for broken strand faults of transmission conductor based on optimized Gabor filter[J].Automation of Electric Power Systems,2011,35(15):78-83.
[17] 王万国,田兵,刘越,等.基于RCNN的无人机巡检图像电力小部件识别研究[J].地球信息科学学报,2017,19(2):256-263.WANG Wanguo,TIAN Bing,LIU Yue,et al.Study on the electrical devices detection in UAV images based on region based convolutional neural networks[J].Journal of Geo-information Science,2017,19(2):256-263.
[18] 陈庆,闫斌,叶润,等.航拍绝缘子卷积神经网络检测及自爆识别研究[J].电子测量与仪器学报,2017,31(6):942-953.CHEN Qing,YAN bin,YE Run,et al.Insulator detection and recognition of explosion fault based on convolutional neural networks[J].Journal of Electronic Measurement and Instrumentation,2017,31(6):942-953.
[19] 李军锋,王钦若,李敏.结合深度学习和随机森林的电力设备图像识别[J].高电压技术,2017,43(11):3705-3711.LI Junfeng,WANG Qinruo,LI Min.Electric equipment image recognition based on deep learning and random forest[J].High Voltage Engineering,2017,43(11):3705-3711.
[20] 汤踊,韩军,魏文力,等.深度学习在输电线路中部件识别与缺陷检测的研究[J].电子测量技术,2018,41(6):60-65.TANG Yong,HAN Jun,WEI Wenli,et al.Research on part recognition and defect detection of transmission line in deep learning[J].Electronic Measurement Technology,2018,41(6):60-65.
[21] 张骥,余娟,汪金礼,等.基于深度学习的输电线路外破图像识别技术[J].计算机系统应用,2018,27(8):176-179.ZHANG Ji,YU Juan,WANG Jinli,et al.Image recognition technology for transmission line external damage based on depth learning[J].Computer System & Applications,2018,27(8):176-179.
[22] DAI J,LI Y,HE K,et al.R-FCN:object detection via region-based fully convolutional networks[C]// Proceedings of the 30th International Conference on Neural Information Processing Systems,December 5-10,2016,Barcelona,Spain:379-387.
[23] REN S,GIRSHICK R,GIRSHICK R,et al.Faster R-CNN:towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2017,39(6):1137.
[24] REDMON J,FARHADI A.YOLOv3:an incremental improvement [EB/OL].[2018-08-01].https://arxiv.org/abs/1804.02767.
[25] SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition [EB/OL].[2018-08-01].https://arxiv.org/abs/1409.1556.
[26] SZEGEDY C,VANHOUCKE V,IOFFE S,et al.Rethinking the inception architecture for computer vision [EB/OL].[2018-08-01].https://arxiv.org/abs/1512.00567.
[27] HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition[EB/OL].[2018-08-01].https://arxiv.org/abs/1512.03385.
[28] XIE S,GIRSHICK R,DOLLAR P,et al.Aggregated residual transformations for deep neural networks [EB/OL].[2018-08-01].https://arxiv.org/abs/1611.05431.
[29] HU J,SHEN L,SUN G.Squeeze-and-excitation networks [EB/OL].[2018-08-01].https://arxiv.org/abs/1709.01507.
[30] REDMON J,DIVVALA S,GIRSHICK R,et al.You only look once:unified,real-time object detection [EB/OL].[2018-08-01].https://arxiv.org/abs/1506.02640.
[31] GIRSHICK R.Fast R-CNN [EB/OL].[2018-08-01].https://arxiv.org/abs/1504.08083.
[32] NEUBECK A,GOOL L V.Efficient non-maximum suppression[C]// 18th International Conference on Pattern Recognition,August 20-24,2006,Hong Kong,China:850-855.
[33] SHRIVASTAVA A,GUPTA A,GIRSHICK R.Training region-based object detectors with online hard example mining [EB/OL].[2018-08-01].https://arxiv.org/abs/1604.03540.
[34] BODLA N,SINGH B,CHELLAPPA R,et al.Soft-NMS:improving object detection with one line of code [EB/OL].[2018-08-01].https://arxiv.org/abs/1704.04503.
[35] LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection[C]// 2017 IEEE International Conference on Computer Vision,October 22-29,2017,Venice,Italy.
[36] CAI Z,VASCONCELOS N.Cascade R-CNN:delving into high quality object detection[EB/OL].[2018-08-01].https://arxiv.org/pdf/1712.00726.pdf.
[37] HE Y,ZHANG X,SAVVIDES M,et al.Softer-NMS:rethinking bounding box regression for accurate object detection[J].[EB/OL].[2018-08-01].https://arxiv.org/abs/1809.08545.