面向多源异构数据源的实际范围索引树索引方法
|
摘要
在发电、输配电、新能源接入及新型能源消费等各个环节产生的大数据需要高效、可靠的数据存储和管理。文中以能源互联网作为虚拟应用场景,提出了集群式多源异构数据存储方案。为满足集群存储中高效查询需求,构建了实际范围索引树(AR-tree)分层索引模型,并提出了面向多源异构数据源的AR-tree索引方法,该方法针对在双层索引模型的基础上,以局部数据索引的实际索引范围为对象,建立全局索引。同时,对所提AR-tree索引方法操作开销进行了综合分析。最后,仿真结果表明AR-tree索引方法能提高查询命中率并提升查询等操作效率。
Reliable data storage and highly efficient management are needed when big data is generated from power generation,transmission,distribution,and new energy access and consumption.Taking the Energy Internet as a virtual application scenario,this paper proposes a cluster storage scheme for multi-source heterogeneous data.In order to meet the needs of efficient query in the cluster storage,a hierarchical indexing model based on the actual range tree(AR-tree)is developed.And the AR-tree indexing method for multi-source heterogeneous data sources is proposed.This method is based on the double layer index model,in which the global index is developed by taking the actual indexing range of local data as the object.Also,the operation cost of the AR-tree indexing method is comprehensively analyzed.Finally,the simulation results show that the ARtree indexing method is able to improve the query hit ratio and efficiency.
Reliable data storage and highly efficient management are needed when big data is generated from power generation,transmission,distribution,and new energy access and consumption.Taking the Energy Internet as a virtual application scenario,this paper proposes a cluster storage scheme for multi-source heterogeneous data.In order to meet the needs of efficient query in the cluster storage,a hierarchical indexing model based on the actual range tree(AR-tree)is developed.And the AR-tree indexing method for multi-source heterogeneous data sources is proposed.This method is based on the double layer index model,in which the global index is developed by taking the actual indexing range of local data as the object.Also,the operation cost of the AR-tree indexing method is comprehensively analyzed.Finally,the simulation results show that the ARtree indexing method is able to improve the query hit ratio and efficiency.
引文
[1]曲朝阳,陈帅,杨帆,等.基于云计算技术的电力大数据预处理属性约简方法[J].电力系统自动化,2014,38(8):67-71.DOI:10.7500/AEPS20130601001.QU Zhaoyang,CHEN Shuai,YANG Fan,et al.An attribute reducing method for electric power big data preprocessing based on cloud computing technology[J].Automation of Electric Power Systems,2014,38(8):67-71.DOI:10.7500/AEPS20130601001.
[2]梅华威,米增强,吴广磊.基于MapReduce模型的间歇性能源海量数据处理技术[J].电力系统自动化,2014,38(15):76-80.DOI:10.7500/AEPS20131215006.MEI Huawei,MI Zengqiang,WU Guanglei.Massive data processing of intermittent energy based on MapReduce model[J].Automation of Electric Power Systems,2014,38(15):76-80.DOI:10.7500/AEPS20131215006.
[3]王德文.基于云计算的电力数据中心基础架构及其关键技术[J].电力系统自动化,2012,36(11):67-71.WANG Dewen.Basic framework and key technology for a new generation of data center in electric power corporation based on cloud computation[J].Automation of Electric Power Systems,2012,36(11):67-71.
[4]FERRAGINA P,VENTURINI R.Compressed cache-oblivious string B-tree[J].Lecture Notes in Computer Science,2013,8125:469-480.
[5]HU Jinglin,QI Deyu.An efficient index structure for range count query based on improved B-tree[J].International Journal of U-and E-Service,Science and Technology,2015,8(4):159-168.
[6]JIN Peiquan,YANG Puyuan,YUE Lihua.Optimizing B+-tree for hybrid storage systems[J].Distributed and Parallel Databases,2015,33(3):449-475.
[7]HAVASI F.An improved B+tree for flash file systems[J].Lecture Notes in Computer Science,2011,6543:297-307.
[8]DEKIHARA H,NAKAMURA Y.An extended technique for R-tree to manage multiple type objects[J].Journal of Computational Methods in Sciences&Engineering,2012,12(1):53-61.
[9]WANG P,RAVISHANKAR C V.Secure and efficient range queries on outsourced databases using Rp-trees[C]//IEEE 29th International Conference on Data Engineering(ICDE),April 8-12,2013,Brisbane,Australia:314-325.
[10]O’NEIL P,CHENG E,GAWLICK D,et al.The logstructured merge-tree(LSM-tree)[J].Acta Informatica Medica,1996,33(4):351-383.
[11]WU Sai,JIANG Dawei,OOI B C,et al.Efficient B-tree based indexing for cloud data processing[J].Proceedings of the VLDB Endowment,2010,3(1/2):1207-1218.
[12]WANG Jinbao,WU Sai,GAO Hong,et al.Indexing multidimensional data in a cloud system[C]//Proceedings of the2010ACM SIGMOD International Conference on Management of Data,June 6-10,2010,Indianapolis,USA:591-602.
[13]AGUILERA M K,GOLAB W M,SHAH M A.A practical scalable distributed B-tree[J].Proceedings of the VLDB Endowment,2008,1(1):598-609.
[14]董朝阳,赵俊华,文福拴,等.从智能电网到能源互联网:基本概念与研究框架[J].电力系统自动化,2014,38(15):1-11.DOI:10.7500/AEPS20140613007.DONG Zhaoyang,ZHAO Junhua,WEN Fushuan,et al.From smart grid to energy internet:basic concept and research framework[J].Automation of Electric Power Systems,2014,38(15):1-11.DOI:10.7500/AEPS20140613007.
[15]黄仁乐,蒲天骄,刘克文,等.城市能源互联网功能体系及应用方案设计[J].电力系统自动化,2015,39(9):26-33.DOI:10.7500/AEPS20141229010.HUANG Renle,PU Tianjiao,LIU Kewen,et al.Design of hierarchy and functions of regional energy internet and its demonstration applications[J].Automation of Electric Power Systems,2015,39(9):26-33.DOI:10.7500/AEPS20141229010.
[2]梅华威,米增强,吴广磊.基于MapReduce模型的间歇性能源海量数据处理技术[J].电力系统自动化,2014,38(15):76-80.DOI:10.7500/AEPS20131215006.MEI Huawei,MI Zengqiang,WU Guanglei.Massive data processing of intermittent energy based on MapReduce model[J].Automation of Electric Power Systems,2014,38(15):76-80.DOI:10.7500/AEPS20131215006.
[3]王德文.基于云计算的电力数据中心基础架构及其关键技术[J].电力系统自动化,2012,36(11):67-71.WANG Dewen.Basic framework and key technology for a new generation of data center in electric power corporation based on cloud computation[J].Automation of Electric Power Systems,2012,36(11):67-71.
[4]FERRAGINA P,VENTURINI R.Compressed cache-oblivious string B-tree[J].Lecture Notes in Computer Science,2013,8125:469-480.
[5]HU Jinglin,QI Deyu.An efficient index structure for range count query based on improved B-tree[J].International Journal of U-and E-Service,Science and Technology,2015,8(4):159-168.
[6]JIN Peiquan,YANG Puyuan,YUE Lihua.Optimizing B+-tree for hybrid storage systems[J].Distributed and Parallel Databases,2015,33(3):449-475.
[7]HAVASI F.An improved B+tree for flash file systems[J].Lecture Notes in Computer Science,2011,6543:297-307.
[8]DEKIHARA H,NAKAMURA Y.An extended technique for R-tree to manage multiple type objects[J].Journal of Computational Methods in Sciences&Engineering,2012,12(1):53-61.
[9]WANG P,RAVISHANKAR C V.Secure and efficient range queries on outsourced databases using Rp-trees[C]//IEEE 29th International Conference on Data Engineering(ICDE),April 8-12,2013,Brisbane,Australia:314-325.
[10]O’NEIL P,CHENG E,GAWLICK D,et al.The logstructured merge-tree(LSM-tree)[J].Acta Informatica Medica,1996,33(4):351-383.
[11]WU Sai,JIANG Dawei,OOI B C,et al.Efficient B-tree based indexing for cloud data processing[J].Proceedings of the VLDB Endowment,2010,3(1/2):1207-1218.
[12]WANG Jinbao,WU Sai,GAO Hong,et al.Indexing multidimensional data in a cloud system[C]//Proceedings of the2010ACM SIGMOD International Conference on Management of Data,June 6-10,2010,Indianapolis,USA:591-602.
[13]AGUILERA M K,GOLAB W M,SHAH M A.A practical scalable distributed B-tree[J].Proceedings of the VLDB Endowment,2008,1(1):598-609.
[14]董朝阳,赵俊华,文福拴,等.从智能电网到能源互联网:基本概念与研究框架[J].电力系统自动化,2014,38(15):1-11.DOI:10.7500/AEPS20140613007.DONG Zhaoyang,ZHAO Junhua,WEN Fushuan,et al.From smart grid to energy internet:basic concept and research framework[J].Automation of Electric Power Systems,2014,38(15):1-11.DOI:10.7500/AEPS20140613007.
[15]黄仁乐,蒲天骄,刘克文,等.城市能源互联网功能体系及应用方案设计[J].电力系统自动化,2015,39(9):26-33.DOI:10.7500/AEPS20141229010.HUANG Renle,PU Tianjiao,LIU Kewen,et al.Design of hierarchy and functions of regional energy internet and its demonstration applications[J].Automation of Electric Power Systems,2015,39(9):26-33.DOI:10.7500/AEPS20141229010.