面向用电数据的HDFS数据读取策略
|
摘要
随着智能电网的发展,数据的访问并发量在不断增加,如何对这些海量的用电数据进行高效读取是现今电力企业亟待解决的问题。文章在基于HDFS读策略的基础上,提出一种面向用电数据的HDFS数据读取策略。综合节点的网络距离、带宽利用率和CPU使用率3种因素,通过评判函数找出性能最优的节点并进行访问。实验验证,该策略可以有效提高数据的读取效率,提升了数据的读性能服务。
With the development of the scale of smart grid, the amount of concurrent data access is also increasing, and how to read these huge amounts of electricity data efficiently is an urgent problem to be solved by power enterprises. An HDFS reading strategy for electricity data based on HDFS read strategy is proposed in this paper. Comprehensive node's network distance, bandwidth utilization and CPU utilization of three factors and find the best performance of the node to visit by the evaluation function. Experiment shows that this strategy can improve the data read efficiency and improve the data read performance service.
With the development of the scale of smart grid, the amount of concurrent data access is also increasing, and how to read these huge amounts of electricity data efficiently is an urgent problem to be solved by power enterprises. An HDFS reading strategy for electricity data based on HDFS read strategy is proposed in this paper. Comprehensive node's network distance, bandwidth utilization and CPU utilization of three factors and find the best performance of the node to visit by the evaluation function. Experiment shows that this strategy can improve the data read efficiency and improve the data read performance service.
引文
[1]张沛,杨华飞,许元斌.电力大数据及其在电网公司的应用[J].中国电机工程学报,2014(S1):85-92.
[2]宋亚奇,周国亮,朱永利.智能电网大数据处理技术现状与挑战[J].电网技术,2013(4):927-935.
[3]胡江溢,祝恩国,杜新纲,等.用电信息采集系统应用现状及发展趋势[J].电力系统自动化,2014(2):131-135.
[4]陈佳.基于灰色优势分析的HDFS数据读取方法方案设计[J].电脑迷,2017(4):165-166.
[5]李强,孙震宇,孙功星.一种面向HDFS的数据随机访问方法[J].计算机工程与应用,2017(10):1-7.
[2]宋亚奇,周国亮,朱永利.智能电网大数据处理技术现状与挑战[J].电网技术,2013(4):927-935.
[3]胡江溢,祝恩国,杜新纲,等.用电信息采集系统应用现状及发展趋势[J].电力系统自动化,2014(2):131-135.
[4]陈佳.基于灰色优势分析的HDFS数据读取方法方案设计[J].电脑迷,2017(4):165-166.
[5]李强,孙震宇,孙功星.一种面向HDFS的数据随机访问方法[J].计算机工程与应用,2017(10):1-7.