基于云计算的智能电网大数据处理平台
|
摘要
提出基于云计算的智能电网大数据处理平台SP-DPP(smart power system big data processing platform in cloud environment)。讨论智能电网大数据处理的数学模型与电网大数据的任务划分方式。SP-DPP云平台由大数据存储与管理模块、任务分配与调度模块、大数据执行模块和客户端模块组成。描述SP-DPP云平台处理智能电网大数据的编码方式,以IEEE118节点的电网作为智能电网大数据处理的案例程序。测试结果表明,针对海量的智能电网潮流计算的状态安全大数据的分析需求,SP-DPP平台具有较好的吞吐量与加速比。
A smart power system big data processing platform in cloud environment called SP-DPP was presented.The mathematic model and sub-task partition of big data processing in smart power system were discussed.The SP-DPP was composed of big data storage and management module,task dispatch and scheduling module,big data executing module and client module.The coding model of big data processing on SP-DPP was also described in detail.To demonstrate the effectiveness of SP-DPP,a serial of simulation experiments in IEEE 118 node power system were done.The results and performance analysis demonstrate that SP-DPP can meet the real time demands in large scale power system big data processing.Satisfied throughput and speedup are also obtained in test results.
A smart power system big data processing platform in cloud environment called SP-DPP was presented.The mathematic model and sub-task partition of big data processing in smart power system were discussed.The SP-DPP was composed of big data storage and management module,task dispatch and scheduling module,big data executing module and client module.The coding model of big data processing on SP-DPP was also described in detail.To demonstrate the effectiveness of SP-DPP,a serial of simulation experiments in IEEE 118 node power system were done.The results and performance analysis demonstrate that SP-DPP can meet the real time demands in large scale power system big data processing.Satisfied throughput and speedup are also obtained in test results.
引文
[1]PENG Mengjing,MA Liming.The application of load flow and parallel calculation in power transmission[J].Journal of Shandong Agricultural University(Natural Science Edition),2017,4(4):597-601(in Chinese).[彭梦晶,马丽明.潮流计算与并行计算在电力传输中的应用[J].山东农业大学学报(自然科学版),2017,4(4):597-601.]
[2]SONG Yaqi,ZHOU Guoliang,ZHU Yongli.Present status and challenges of big data processing in smart grid[J].Power System Technology,2013,37(4):927-935(in Chinese).[宋亚奇,周国亮,朱永利.智能电网大数据处理技术现状与挑战[J].电网技术,2013,37(4):927-935.]
[3]Khorsandi A,Hosseinian SH,Ghazanfari A.Modified artificial bee colony algorithm based on fuzzy multi-objective technique for optimal power flow problem[J].Electric Power Systems Research,2013,95(95):206-213.
[4]ZHU Xueling,ZHANG Cuiying,ZHAO Chenpeng,et al.Power system optimal power flow calculation based on Newton’s method[J].Journal of North China University of Water Resources and Electric Power(Natural Science Edition),2014,35(3):71-74(in Chinese).[朱雪凌,张翠影,赵臣鹏,等.基于牛顿法的电力系统最优潮流计算[J].华北水利水电大学学报(自然科学版),2014,35(3):71-74.]
[5]Spangler R,Shoults R.Power generation,operation,and control[J].IEEE Power&Energy Magazine,2014,12(4):90-93.
[6]Zheng Y,Capra L,Wolfson O,et al.Urban computing:Concepts,methodologies,and applications[J].ACM Transaction on Intelligent Systems and Technology,2014,1(1):1-9.
[7]QU Zhaoyang,CHEN Shuai,YANG Fan,et al.An attribute reducing method for electric power big data preprocessing based on cloud computing technology[J].Automation of Electric Power Systems,2014,38(8):67-71(in Chinese).[曲朝阳,陈帅,杨帆,等.基于云计算技术的电力大数据预处理属性约简方法[J].电力系统自动化,2014,38(8):67-71.]
[8]WANG Cheng,WEI Qinglai,ZHAO Dongbin,et al.Databased adaptive optimal control of smart grid power[J].Control Engineering of China,2014,21(5):753-759(in Chinese).[王澄,魏庆来,赵冬斌,等.基于数据的智能电网电能自适应优化调控[J].控制工程,2014,21(5):753-759.]
[9]China cloud computing network[EB/OL].[2017-03-01].http://www.chinacloud.cn/(in Chinese).[中国云计算网web site 2017[EB/OL].[2017-03-01].http://www.chinacloud.cn/.]
[10]Marozzo F,Talia D,Trunfio P.P2P-MapReduce:Parallel data processing in dynamic cloud environments[J].Journal of Computer&System Sciences,2012,78(5):1382-1402.
[11]Li F,Ooi BC,zsu MT,et al.Distributed data management using MapReduce[J].ACM Computing Surveys,2014,46(3):1-42.
[12]WU Lizhong,XU Cheng.Ower system power flow calculation with UPFC[J].Yunnan Electric Power Systems,2016,44(4):107-109(in Chinese).[武历忠,徐诚.电力系统潮流计算[J].云南电力技术,2016,44(4):107-109.]
[13]FANG Meng.Research and advancements of power flow computation[J].China New Telecommunications,2015(17):26(in Chinese).[房盟.电力系统潮流计算研究现状及发展展望[J].中国新通信,2015(17):26.]
[14]XU Yuanbin,ZHONG Xiaoqiang,WANG Dan,et al.MapReduce model-based parallelized anomaly detection of electric power data[J].Information Research,2014,202(8):74-78(in Chinese).[许元斌,钟小强,王丹,等.基于MapReduce模型的电力数据并行化的异常检测[J].情报探索,2014,202(8):74-78.]
[15]WANG Yuwei,LIU Min,FANG Bingyi,et al.Study on virtual computing resource scheduling for network performance optimization[J].Journal on Communications,2016,37(8):105-118(in Chinese).[王煜炜,刘敏,房秉毅,等.面向网络性能优化的虚拟计算资源调度机制研究[J].通信学报,2016,37(8):105-118.]
[16]HU Haiyang,LIU Runhua,HU Hua.Multi-objective optimization for task scheduling in mobile cloud computing[J].Journal of Computer Research and Development,2017,54(9):1909-1919(in Chinese).[胡海洋,刘润华,胡华.移动云计算环境下任务调度的多目标优化方法[J].计算机研究与发展,2017,54(9):1909-1919.]
[17]CUI Yong,SONG Jian,MIAO Congcong,et al.Mobile cloud computing research progress and trends[J].Chinese Journal of Computers,2017,40(2):273-295(in Chinese).[崔勇,宋健,缪葱葱,等.移动云计算研究进展与趋势[J].计算机学报,2017,40(2):273-295.]
[18]Tang B,He H,Fedak G.HybridMR:A new approach for hybrid MapReduce combining desktop grid and cloud infrastructures[J].Concurrency&Computation Practice&Experience,2015,27(16):4140-4155.
[19]WANG Qiang,LI Xiongfei,WANG Jing.A data placement and task scheduling algorithm in cloud computing[J].Journal of Computer Research and Development,2014,51(11):2416-2426(in Chinese).[王强,李雄飞,王婧.云计算中的数据放置与任务调度算法[J].计算机研究与发展,2014,51(11):2416-2426.]
[20]Hadoop open source web site 2017[EB/OL].[2017-03-01].http://hadoop.apache.org/.
[21]WU Xiaoping,ZHOU Zhou,LI Hongcheng.Research and implementation on network traffic anomaly detection without guidance learning with Spark[J].Netinfo Security,2016(6):1-7(in Chinese).[吴晓平,周舟,李洪成.Spark框架下基于无指导学习环境的网络流量异常检测研究与实现[J].信息网络安全,2016(6):1-7.]
[2]SONG Yaqi,ZHOU Guoliang,ZHU Yongli.Present status and challenges of big data processing in smart grid[J].Power System Technology,2013,37(4):927-935(in Chinese).[宋亚奇,周国亮,朱永利.智能电网大数据处理技术现状与挑战[J].电网技术,2013,37(4):927-935.]
[3]Khorsandi A,Hosseinian SH,Ghazanfari A.Modified artificial bee colony algorithm based on fuzzy multi-objective technique for optimal power flow problem[J].Electric Power Systems Research,2013,95(95):206-213.
[4]ZHU Xueling,ZHANG Cuiying,ZHAO Chenpeng,et al.Power system optimal power flow calculation based on Newton’s method[J].Journal of North China University of Water Resources and Electric Power(Natural Science Edition),2014,35(3):71-74(in Chinese).[朱雪凌,张翠影,赵臣鹏,等.基于牛顿法的电力系统最优潮流计算[J].华北水利水电大学学报(自然科学版),2014,35(3):71-74.]
[5]Spangler R,Shoults R.Power generation,operation,and control[J].IEEE Power&Energy Magazine,2014,12(4):90-93.
[6]Zheng Y,Capra L,Wolfson O,et al.Urban computing:Concepts,methodologies,and applications[J].ACM Transaction on Intelligent Systems and Technology,2014,1(1):1-9.
[7]QU Zhaoyang,CHEN Shuai,YANG Fan,et al.An attribute reducing method for electric power big data preprocessing based on cloud computing technology[J].Automation of Electric Power Systems,2014,38(8):67-71(in Chinese).[曲朝阳,陈帅,杨帆,等.基于云计算技术的电力大数据预处理属性约简方法[J].电力系统自动化,2014,38(8):67-71.]
[8]WANG Cheng,WEI Qinglai,ZHAO Dongbin,et al.Databased adaptive optimal control of smart grid power[J].Control Engineering of China,2014,21(5):753-759(in Chinese).[王澄,魏庆来,赵冬斌,等.基于数据的智能电网电能自适应优化调控[J].控制工程,2014,21(5):753-759.]
[9]China cloud computing network[EB/OL].[2017-03-01].http://www.chinacloud.cn/(in Chinese).[中国云计算网web site 2017[EB/OL].[2017-03-01].http://www.chinacloud.cn/.]
[10]Marozzo F,Talia D,Trunfio P.P2P-MapReduce:Parallel data processing in dynamic cloud environments[J].Journal of Computer&System Sciences,2012,78(5):1382-1402.
[11]Li F,Ooi BC,zsu MT,et al.Distributed data management using MapReduce[J].ACM Computing Surveys,2014,46(3):1-42.
[12]WU Lizhong,XU Cheng.Ower system power flow calculation with UPFC[J].Yunnan Electric Power Systems,2016,44(4):107-109(in Chinese).[武历忠,徐诚.电力系统潮流计算[J].云南电力技术,2016,44(4):107-109.]
[13]FANG Meng.Research and advancements of power flow computation[J].China New Telecommunications,2015(17):26(in Chinese).[房盟.电力系统潮流计算研究现状及发展展望[J].中国新通信,2015(17):26.]
[14]XU Yuanbin,ZHONG Xiaoqiang,WANG Dan,et al.MapReduce model-based parallelized anomaly detection of electric power data[J].Information Research,2014,202(8):74-78(in Chinese).[许元斌,钟小强,王丹,等.基于MapReduce模型的电力数据并行化的异常检测[J].情报探索,2014,202(8):74-78.]
[15]WANG Yuwei,LIU Min,FANG Bingyi,et al.Study on virtual computing resource scheduling for network performance optimization[J].Journal on Communications,2016,37(8):105-118(in Chinese).[王煜炜,刘敏,房秉毅,等.面向网络性能优化的虚拟计算资源调度机制研究[J].通信学报,2016,37(8):105-118.]
[16]HU Haiyang,LIU Runhua,HU Hua.Multi-objective optimization for task scheduling in mobile cloud computing[J].Journal of Computer Research and Development,2017,54(9):1909-1919(in Chinese).[胡海洋,刘润华,胡华.移动云计算环境下任务调度的多目标优化方法[J].计算机研究与发展,2017,54(9):1909-1919.]
[17]CUI Yong,SONG Jian,MIAO Congcong,et al.Mobile cloud computing research progress and trends[J].Chinese Journal of Computers,2017,40(2):273-295(in Chinese).[崔勇,宋健,缪葱葱,等.移动云计算研究进展与趋势[J].计算机学报,2017,40(2):273-295.]
[18]Tang B,He H,Fedak G.HybridMR:A new approach for hybrid MapReduce combining desktop grid and cloud infrastructures[J].Concurrency&Computation Practice&Experience,2015,27(16):4140-4155.
[19]WANG Qiang,LI Xiongfei,WANG Jing.A data placement and task scheduling algorithm in cloud computing[J].Journal of Computer Research and Development,2014,51(11):2416-2426(in Chinese).[王强,李雄飞,王婧.云计算中的数据放置与任务调度算法[J].计算机研究与发展,2014,51(11):2416-2426.]
[20]Hadoop open source web site 2017[EB/OL].[2017-03-01].http://hadoop.apache.org/.
[21]WU Xiaoping,ZHOU Zhou,LI Hongcheng.Research and implementation on network traffic anomaly detection without guidance learning with Spark[J].Netinfo Security,2016(6):1-7(in Chinese).[吴晓平,周舟,李洪成.Spark框架下基于无指导学习环境的网络流量异常检测研究与实现[J].信息网络安全,2016(6):1-7.]