云平台下时间序列数据并行化排列熵特征提取方法
|
摘要
随着高级量测体系和各类监控系统的大规模建设发展,时间序列数据的规模呈指数级增长,在智能电网大数据中占有较大的比重。时间序列数据的特征提取是影响数据挖掘质量的关键步骤,在大数据背景下,传统的特征提取算法已无法满足海量数据处理的需求。结合云计算平台和MaxCompute大数据处理技术,设计实现了时间序列数据的表存储方法和并行化的时间序列数据排列熵特征提取算法。在云计算平台上采用不同规模的数据集对并行化算法进行测试,验证了并行化排列熵算法的正确性和高性能。
With the large-scale construction and development of AMI(Advanced Metering Infrastructure) and various monitoring systems,the size of time series data grows exponentially,which occupies a large proportion in the smart grid big data. The feature extraction of time series data is a key step that affects the quality of data mining. Traditio-nal feature extraction algorithms can no longer meet the requirements of mass data processing in the context of big data. The table storage method and feature extraction algorithm based on parallel permutation entropy are designed and implemented for time series data by combining with the cloud computing platform and MaxCompute big data processing technology. Different scale data sets are tested on the cloud computing platform,and results verify the accu-racy and high performance of the parallel permutation entropy algorithm.
With the large-scale construction and development of AMI(Advanced Metering Infrastructure) and various monitoring systems,the size of time series data grows exponentially,which occupies a large proportion in the smart grid big data. The feature extraction of time series data is a key step that affects the quality of data mining. Traditio-nal feature extraction algorithms can no longer meet the requirements of mass data processing in the context of big data. The table storage method and feature extraction algorithm based on parallel permutation entropy are designed and implemented for time series data by combining with the cloud computing platform and MaxCompute big data processing technology. Different scale data sets are tested on the cloud computing platform,and results verify the accu-racy and high performance of the parallel permutation entropy algorithm.
引文
[1] 宋亚奇,周国亮,朱永利. 智能电网大数据处理技术现状与挑战[J]. 电网技术,2013,37(4):927-935.SONG Yaqi,ZHOU Guoliang,ZHU Yongli. Present status and challenges of big data processing in smart grid[J]. Power System Technology,2013,37(4):927-935.
[2] BANDT C,POMPE B. Permutation entropy:a natural complexity measure for time series[J]. Physical Review Letters,2002,88(17):174102.
[3] 刘永斌,龙潜,冯志华,等. 一种非平稳、非线性振动信号检测方法的研究[J]. 振动与冲击,2007,26(12):131-134.LIU Yongbin,LONG Qian,FENG Zhihua,et al. Detection method for nonlinear and non-stationary signals[J]. Journal of Vibration and Shock,2007,26(12):131-134.
[4] 冯辅周,饶国强,司爱威,等. 排列熵算法研究及其在振动信号突变检测中的应用[J]. 振动工程学报,2012,25(2):221-224.FENG Fuzhou,RAO Guoqiang,SI Aiwei,et al. Research and application of the arithmetic of PE in testing the sudden change of vibration signal[J]. Journal of Vibration Engineering,2012,25(2):221-224.
[5] 张蒙,朱永利,张宁,等. 基于变分模态分解和多尺度排列熵的变压器局部放电信号特征提取[J]. 华北电力大学学报(自然科学版),2016,43(6):31-37.ZHANG Meng,ZHU Yongli,ZHANG Ning,et al. Feature extraction of transformer partial discharge signals based on varitional mode decomposition and multi-scale permutation entropy[J]. Journal of North China Electric Power University(Natural Science Edition),2016,43(6):31-37.
[6] NICOLAOU N,GEORGIOU J. Detection of epileptic electroencephalogram based on permutation entropy and support vector machines[J]. Expert Systems with Applications,2012,39(1):202-209.
[7] SUN X L,ZOU Y,NIKIFOROVA V,et al. The complexity of gene expression dynamics revealed by permutation entropy[J]. BMC Bio-informatics,2010,11(1):607-621.
[8] 郝成元,吴绍洪,李双成. 排列熵应用于气候复杂性度量[J]. 地理研究,2007,26(1):46-52.HAO Chengyuan,WU Shaohong,LI Shuangcheng. Measurement of climate complexity using permutation entropy[J]. Geographical Research,2007,26(1):46-52.
[9] LUO Y J,YE X M,FAN Q G. Texture image analysis based on discrete Fourier transform and permutation entropy[J]. Electronic Science and Technology,2011,24(1):9-11.
[10] 吴秀良,范影乐,钱诚,等. 基于排列组合熵的语音端点检测技术研究[J]. 计算机工程与应用,2008,44(1):240-242.WU Xiuliang,FAN Yingle,QIAN Cheng,et al. Application of permutation entropy measure in detecting speech[J]. Computer Engineering and Applications,2008,44(1):240-242.
[11] 李从善,刘天琪,李兴源,等. 基于排列熵算法的电力系统故障信号分析[J]. 电子科技大学学报,2015,44(2):233-238.LI Congshan,LIU Tianqi,LI Xingyuan,et al. Power system fault sig-nal analysis based on permutation entropy algorithm[J]. Journal of University of Electronic Science and Technology of China,2015,44(2):233-238.
[12] 何军娜. 基于小波及排列熵奇异性检测的行波故障测距研究[D]. 南昌:华东交通大学,2015.HE Junna. Research on transmission line traveling wave fault location based on wavelet transform and the permutation entropy[D]. Nanchang:East China Jiaotong University,2015.
[13] 姜媛媛,刘朋,王康,等. 基于变分模态分解和排列熵的输电线路故障诊断[J]. 电子测量与仪器学报,2017,31(7):1025-1030. JIANG Yuanyuan,LIU Peng,WANG Kang,et al. Fault diagnosis of transmission lines based on variational mode decomposition and permutation entropy[J]. Journal of Electronic Measurement and Instrumentation,2017,31(7):1025-1030.
[14] 宋亚奇,刘树仁,朱永利,等. 电力设备状态高速采样数据的云存储技术研究[J]. 电力自动化设备,2013,33(10):150-156.SONG Yaqi,LIU Shuren,ZHU Yongli,et al. Cloud storage of power equipment state data sampled with high speed[J]. Electric Power Automation Equipment,2013,33(10):150-156.
[15] 宋亚奇,周国亮,朱永利,等. 云平台下输变电设备状态监测大数据存储优化与并行处理[J]. 中国电机工程学报,2015,35(2):255-267.SONG Yaqi,ZHOU Guoliang,ZHU Yongli,et al. Storage optimi-zation and parallel processing of condition monitoring big data of transmission and transforming equipment based on cloud platform[J]. Proceedings of the CSEE,2015,35(2):255-267.
[16] 朱永利,李莉,宋亚奇,等. ODPS平台下的电力设备监测大数据存储与并行处理方法[J]. 电工技术学报,2017,32(9):199-210.ZHU Yongli,LI Li,SONG Yaqi,et al. Storage and parallel proces-sing of big data of power equipment condition monitoring on ODPS platform[J]. Transactions of China Electrotechnical Society,2017,32(9):199-210.
[2] BANDT C,POMPE B. Permutation entropy:a natural complexity measure for time series[J]. Physical Review Letters,2002,88(17):174102.
[3] 刘永斌,龙潜,冯志华,等. 一种非平稳、非线性振动信号检测方法的研究[J]. 振动与冲击,2007,26(12):131-134.LIU Yongbin,LONG Qian,FENG Zhihua,et al. Detection method for nonlinear and non-stationary signals[J]. Journal of Vibration and Shock,2007,26(12):131-134.
[4] 冯辅周,饶国强,司爱威,等. 排列熵算法研究及其在振动信号突变检测中的应用[J]. 振动工程学报,2012,25(2):221-224.FENG Fuzhou,RAO Guoqiang,SI Aiwei,et al. Research and application of the arithmetic of PE in testing the sudden change of vibration signal[J]. Journal of Vibration Engineering,2012,25(2):221-224.
[5] 张蒙,朱永利,张宁,等. 基于变分模态分解和多尺度排列熵的变压器局部放电信号特征提取[J]. 华北电力大学学报(自然科学版),2016,43(6):31-37.ZHANG Meng,ZHU Yongli,ZHANG Ning,et al. Feature extraction of transformer partial discharge signals based on varitional mode decomposition and multi-scale permutation entropy[J]. Journal of North China Electric Power University(Natural Science Edition),2016,43(6):31-37.
[6] NICOLAOU N,GEORGIOU J. Detection of epileptic electroencephalogram based on permutation entropy and support vector machines[J]. Expert Systems with Applications,2012,39(1):202-209.
[7] SUN X L,ZOU Y,NIKIFOROVA V,et al. The complexity of gene expression dynamics revealed by permutation entropy[J]. BMC Bio-informatics,2010,11(1):607-621.
[8] 郝成元,吴绍洪,李双成. 排列熵应用于气候复杂性度量[J]. 地理研究,2007,26(1):46-52.HAO Chengyuan,WU Shaohong,LI Shuangcheng. Measurement of climate complexity using permutation entropy[J]. Geographical Research,2007,26(1):46-52.
[9] LUO Y J,YE X M,FAN Q G. Texture image analysis based on discrete Fourier transform and permutation entropy[J]. Electronic Science and Technology,2011,24(1):9-11.
[10] 吴秀良,范影乐,钱诚,等. 基于排列组合熵的语音端点检测技术研究[J]. 计算机工程与应用,2008,44(1):240-242.WU Xiuliang,FAN Yingle,QIAN Cheng,et al. Application of permutation entropy measure in detecting speech[J]. Computer Engineering and Applications,2008,44(1):240-242.
[11] 李从善,刘天琪,李兴源,等. 基于排列熵算法的电力系统故障信号分析[J]. 电子科技大学学报,2015,44(2):233-238.LI Congshan,LIU Tianqi,LI Xingyuan,et al. Power system fault sig-nal analysis based on permutation entropy algorithm[J]. Journal of University of Electronic Science and Technology of China,2015,44(2):233-238.
[12] 何军娜. 基于小波及排列熵奇异性检测的行波故障测距研究[D]. 南昌:华东交通大学,2015.HE Junna. Research on transmission line traveling wave fault location based on wavelet transform and the permutation entropy[D]. Nanchang:East China Jiaotong University,2015.
[13] 姜媛媛,刘朋,王康,等. 基于变分模态分解和排列熵的输电线路故障诊断[J]. 电子测量与仪器学报,2017,31(7):1025-1030. JIANG Yuanyuan,LIU Peng,WANG Kang,et al. Fault diagnosis of transmission lines based on variational mode decomposition and permutation entropy[J]. Journal of Electronic Measurement and Instrumentation,2017,31(7):1025-1030.
[14] 宋亚奇,刘树仁,朱永利,等. 电力设备状态高速采样数据的云存储技术研究[J]. 电力自动化设备,2013,33(10):150-156.SONG Yaqi,LIU Shuren,ZHU Yongli,et al. Cloud storage of power equipment state data sampled with high speed[J]. Electric Power Automation Equipment,2013,33(10):150-156.
[15] 宋亚奇,周国亮,朱永利,等. 云平台下输变电设备状态监测大数据存储优化与并行处理[J]. 中国电机工程学报,2015,35(2):255-267.SONG Yaqi,ZHOU Guoliang,ZHU Yongli,et al. Storage optimi-zation and parallel processing of condition monitoring big data of transmission and transforming equipment based on cloud platform[J]. Proceedings of the CSEE,2015,35(2):255-267.
[16] 朱永利,李莉,宋亚奇,等. ODPS平台下的电力设备监测大数据存储与并行处理方法[J]. 电工技术学报,2017,32(9):199-210.ZHU Yongli,LI Li,SONG Yaqi,et al. Storage and parallel proces-sing of big data of power equipment condition monitoring on ODPS platform[J]. Transactions of China Electrotechnical Society,2017,32(9):199-210.