基于孤立森林算法的电力调度流数据异常检测方法
|
摘要
调度是电力系统安全运行的保障。针对具有"概念漂移"特点的调度监测流数据,基于离线数据分析或简单阈值判定的异常检测方法,存在与生产系统实时运行状态结合不紧密、依赖专家经验等问题。提出了一种基于孤立森林算法的电力调度流数据异常检测方法,利用历史数据集训练构建多个子森林异常检测器,组成基森林异常检测器;据此,在线根据滑动窗口中数据的异常情况及缓冲区数据量大小,触发检测器更新。提出一种根据异常偏差率大小筛选子森林异常检测器的更新策略,解决因模型随机更新导致异常检测器整体性能下降的问题。以服务器和某省级电网调度中心业务流数据集作为训练与测试样本,验证了所提方法在异常检测查全率及查准率等综合性能上的先进性及其在实际系统应用中的可行性。
Dispatching is the guarantee of safe operation of power system. For scheduling monitoring flow data with "concept drift" feature, there are many issues in some aspects when using traditional offline data analysis or simple threshold judgement of anomaly detection methods, such as defective tightness between existing production systems and real-time operation state, excessive dependence on expert experience,etc. This paper puts forward an algorithm based on isolated forest data anomaly detection method of electric power dispatching flow, using historical data set to train and build many child forest anomaly detectors and compose base forest anomaly detector. Therefore, the detector update is triggered online according to abnormal data situation in sliding window and data size in cache area. An update strategy selecting sub-forest abnormal detector according to the size of abnormal deviation rate is proposed to solve dropping issue of the abnormal detector performance caused by random update.Using the flow of business data set from servers and provincial power grid dispatching centers as training and testing samples,the proposed method is verified on its advancement on comprehensive performance of anomaly detection such as recall and precision, feasible in practical system application.
Dispatching is the guarantee of safe operation of power system. For scheduling monitoring flow data with "concept drift" feature, there are many issues in some aspects when using traditional offline data analysis or simple threshold judgement of anomaly detection methods, such as defective tightness between existing production systems and real-time operation state, excessive dependence on expert experience,etc. This paper puts forward an algorithm based on isolated forest data anomaly detection method of electric power dispatching flow, using historical data set to train and build many child forest anomaly detectors and compose base forest anomaly detector. Therefore, the detector update is triggered online according to abnormal data situation in sliding window and data size in cache area. An update strategy selecting sub-forest abnormal detector according to the size of abnormal deviation rate is proposed to solve dropping issue of the abnormal detector performance caused by random update.Using the flow of business data set from servers and provincial power grid dispatching centers as training and testing samples,the proposed method is verified on its advancement on comprehensive performance of anomaly detection such as recall and precision, feasible in practical system application.
引文
[1]张东霞,苗新,刘丽平,等.智能电网大数据技术发展研究[J].中国电机工程学报,2015,35(1):2-12.Zhang Dongxia,Miao Xin,Liu Liping,et al.Research on development strategy for smart grid big data[J].Proceedings of the CSEE,2015,35(1):2-12(in Chinese).
[2]赵良,李立理,何博,等.适合我国国情的智能电网评价指标体系及计算方法[J].电网技术,2015,39(12):3520-3528.Zhao Liang,Li Lili,He Bo,et al.Study on smart grid's evaluation index system and calculation method suited to China's Situation[J].Power System Technology,2015,39(12):3520-3528(in Chinese).
[3]刘铠诚,何光宇,王彬,等.调度中心智能调度业务成熟度模型及其应用[J].电力系统自动化,2012,36(13):1-7.Liu Kaicheng,He Guangyu,Wang Bin,et al.A smart dispatching business maturity model of dispatching center and its application[J].Automation of Electric Power Systems,2012,36(13):1-7(in Chinese).
[4]王斌琪,王海霞,徐鹏,等.电网运行趋势实时安全评估方法[J].电网技术,2015,39(2):478-485.Wang Binqi,Wang Haixia,Xu Peng,et al.Real-time security assessment methods of power grid operation trends[J].Power System Technology,2015,39(2):478-485(in Chinese).
[5]秦璇.电力统计数据的质量评估及其异常检测方法研究[D].长沙:长沙理工大学,2013.
[6]陈凌,韩伟,张经炜.基于数据融合的光伏组件故障诊断[J].电网技术,2017,41(6):1864-1872.Chen Ling,Han Wei,Zhang Jingwei.V module fault diagnosis based on data fusion[J].Power System Technology,2017,41(6):1864-1872(in Chinese).
[7]颜永龙,李剑,李辉,等.采用信息熵和组合模型的风电机组异常检测方法[J].电网技术,2015,39(3):737-743.Yan Yonglong,Li Jian,Li Hui,et al.A wind turbine anomaly detection method based on information entropy and combination model[J].Power System Technology,2015,39(3):737-743(in Chinese).
[8]崔江,唐军祥,张卓然,等.基于极限学习机的航空发电机旋转整流器快速故障分类方法研究[J].中国电机工程学报,2018,38(8):2458-2466.Cui Jiang,Tang Junxiang,Zhang Zhuoran,et al.Fast fault classification method research of aircraft generator rotating rectifier based on extreme learning machine[J].Proceedings of the CSEE,2018,38(8):2458-2466(in Chinese).
[9]赵加奎,杨国凤,沐连顺,等.数据流技术在电网自动化中的应用研究[J].电网技术,2011,35(8):6-11.Zhao Jiakui,Yang Guofeng,Mu Lianshun,et al.Research on the application of the data stream technology in grid automation[J].Power System Technology,2011,35(8):6-11(in Chinese).
[10]王桂兰,周国亮,赵洪山,等.大规模用电数据流的快速聚类和异常检测技术[J].电力系统自动化,2016,40(24):27-33.Wang Guilan,Zhou Guoliang,Zhao Hongshan,et al.Fast clustering and anomaly detection technique for large-scale power data stream[J].Automation of Electric Power Systems,2016,40(24):27-33(in Chinese).
[11]王德文,杨力平.智能电网大数据流式处理方法与状态监测异常检测[J].电力系统自动化,2016,40(14):122-128.Wang Dewen,Yang Liping.Stream processing method and condition monitoring anomaly detection for big data in smart grid[J].Automation of Electric Power Systems,2016,40(14):122-128(in Chinese).
[12]张友强,寇凌峰,盛万兴,等.配电变压器运行状态评估的大数据分析方法[J].电网技术,2016,40(3):768-773.Zhang Youqiang,Kou Lingfeng,Sheng Wanxing,et al.Big data analytical method for operating state assessment of distribution transformer[J].Power System Technology,2016,40(3):768-773(in Chinese).
[13]Krawczyk B,Cano A.Online ensemble learning with abstaining classifiers for drifting and noisy data streams[J].Applied Soft Computing,2018,68:677-692.
[14]Liu F T,Kai M T,Zhou Z H.Isolation forest[C]//Eighth IEEEInternational Conference on Data Mining.IEEE Computer Society,2008:413-422.
[15]丁智国.流数据在线异常检测方法研究[D].上海:上海大学,2015.
[16]García S,Zhang Z L,Altalhi A,et al.Dynamic ensemble selection for multi-class imbalanced datasets[J].Information Sciences,2018,445:22-37.
[17]季一木,张永潘,郎贤波,等.面向流数据的决策树分类算法并行化[J].计算机研究与发展,2017,54(9):1945-1957.Ji Yimu,Zhang Yongpan,Lang Xianbo,et al.Parallel of decision tree classification algorithm for stream data[J].Journal of Computer Research and Development,2017,54(9):1945-1957(in Chinese)
[18]严英杰,盛戈皞,陈玉峰,等.基于大数据分析的输变电设备状态数据异常检测方法[J].中国电机工程学报,2015,35(1):52-59.Yan Yingjie,Sheng Gehao,Chen Yufeng,et al.An method for anomaly detection of state information of power equipment based on big data analysis[J].Proceedings of the CSEE,2015,35(1):52-59(in Chinese).
[19]吕朋蓬,赵晋泉,李端超,等.电网运行状态评价指标体系与综合评价方法[J].电网技术,2015,39(8):2245-2252.LüPengpeng,Zhao Jinquan,Li Duanchao,et al.An assessment index system for power grid operation status and corresponding synthetic assessment method[J].Power System Technology,2015,39(8):2245-2252(in Chinese).
[20]Liu F T,Kai M T,Zhou Z H.Isolation-based anomaly detection[J].ACM Transactions on Knowledge Discovery from Data,2012,6(1):1-39.
[21]周志华.机器学习[M].北京:清华大学出版社,2016.
[2]赵良,李立理,何博,等.适合我国国情的智能电网评价指标体系及计算方法[J].电网技术,2015,39(12):3520-3528.Zhao Liang,Li Lili,He Bo,et al.Study on smart grid's evaluation index system and calculation method suited to China's Situation[J].Power System Technology,2015,39(12):3520-3528(in Chinese).
[3]刘铠诚,何光宇,王彬,等.调度中心智能调度业务成熟度模型及其应用[J].电力系统自动化,2012,36(13):1-7.Liu Kaicheng,He Guangyu,Wang Bin,et al.A smart dispatching business maturity model of dispatching center and its application[J].Automation of Electric Power Systems,2012,36(13):1-7(in Chinese).
[4]王斌琪,王海霞,徐鹏,等.电网运行趋势实时安全评估方法[J].电网技术,2015,39(2):478-485.Wang Binqi,Wang Haixia,Xu Peng,et al.Real-time security assessment methods of power grid operation trends[J].Power System Technology,2015,39(2):478-485(in Chinese).
[5]秦璇.电力统计数据的质量评估及其异常检测方法研究[D].长沙:长沙理工大学,2013.
[6]陈凌,韩伟,张经炜.基于数据融合的光伏组件故障诊断[J].电网技术,2017,41(6):1864-1872.Chen Ling,Han Wei,Zhang Jingwei.V module fault diagnosis based on data fusion[J].Power System Technology,2017,41(6):1864-1872(in Chinese).
[7]颜永龙,李剑,李辉,等.采用信息熵和组合模型的风电机组异常检测方法[J].电网技术,2015,39(3):737-743.Yan Yonglong,Li Jian,Li Hui,et al.A wind turbine anomaly detection method based on information entropy and combination model[J].Power System Technology,2015,39(3):737-743(in Chinese).
[8]崔江,唐军祥,张卓然,等.基于极限学习机的航空发电机旋转整流器快速故障分类方法研究[J].中国电机工程学报,2018,38(8):2458-2466.Cui Jiang,Tang Junxiang,Zhang Zhuoran,et al.Fast fault classification method research of aircraft generator rotating rectifier based on extreme learning machine[J].Proceedings of the CSEE,2018,38(8):2458-2466(in Chinese).
[9]赵加奎,杨国凤,沐连顺,等.数据流技术在电网自动化中的应用研究[J].电网技术,2011,35(8):6-11.Zhao Jiakui,Yang Guofeng,Mu Lianshun,et al.Research on the application of the data stream technology in grid automation[J].Power System Technology,2011,35(8):6-11(in Chinese).
[10]王桂兰,周国亮,赵洪山,等.大规模用电数据流的快速聚类和异常检测技术[J].电力系统自动化,2016,40(24):27-33.Wang Guilan,Zhou Guoliang,Zhao Hongshan,et al.Fast clustering and anomaly detection technique for large-scale power data stream[J].Automation of Electric Power Systems,2016,40(24):27-33(in Chinese).
[11]王德文,杨力平.智能电网大数据流式处理方法与状态监测异常检测[J].电力系统自动化,2016,40(14):122-128.Wang Dewen,Yang Liping.Stream processing method and condition monitoring anomaly detection for big data in smart grid[J].Automation of Electric Power Systems,2016,40(14):122-128(in Chinese).
[12]张友强,寇凌峰,盛万兴,等.配电变压器运行状态评估的大数据分析方法[J].电网技术,2016,40(3):768-773.Zhang Youqiang,Kou Lingfeng,Sheng Wanxing,et al.Big data analytical method for operating state assessment of distribution transformer[J].Power System Technology,2016,40(3):768-773(in Chinese).
[13]Krawczyk B,Cano A.Online ensemble learning with abstaining classifiers for drifting and noisy data streams[J].Applied Soft Computing,2018,68:677-692.
[14]Liu F T,Kai M T,Zhou Z H.Isolation forest[C]//Eighth IEEEInternational Conference on Data Mining.IEEE Computer Society,2008:413-422.
[15]丁智国.流数据在线异常检测方法研究[D].上海:上海大学,2015.
[16]García S,Zhang Z L,Altalhi A,et al.Dynamic ensemble selection for multi-class imbalanced datasets[J].Information Sciences,2018,445:22-37.
[17]季一木,张永潘,郎贤波,等.面向流数据的决策树分类算法并行化[J].计算机研究与发展,2017,54(9):1945-1957.Ji Yimu,Zhang Yongpan,Lang Xianbo,et al.Parallel of decision tree classification algorithm for stream data[J].Journal of Computer Research and Development,2017,54(9):1945-1957(in Chinese)
[18]严英杰,盛戈皞,陈玉峰,等.基于大数据分析的输变电设备状态数据异常检测方法[J].中国电机工程学报,2015,35(1):52-59.Yan Yingjie,Sheng Gehao,Chen Yufeng,et al.An method for anomaly detection of state information of power equipment based on big data analysis[J].Proceedings of the CSEE,2015,35(1):52-59(in Chinese).
[19]吕朋蓬,赵晋泉,李端超,等.电网运行状态评价指标体系与综合评价方法[J].电网技术,2015,39(8):2245-2252.LüPengpeng,Zhao Jinquan,Li Duanchao,et al.An assessment index system for power grid operation status and corresponding synthetic assessment method[J].Power System Technology,2015,39(8):2245-2252(in Chinese).
[20]Liu F T,Kai M T,Zhou Z H.Isolation-based anomaly detection[J].ACM Transactions on Knowledge Discovery from Data,2012,6(1):1-39.
[21]周志华.机器学习[M].北京:清华大学出版社,2016.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |