云存储系统可扩展性评测研究
|
摘要
云存储系统中管理大规模数据的能力是云存储系统可拓展性的重要组成部分,而超大容量的测试需求是云存储系统可扩展性评测的难点。针对此问题,分别在水平扩展和垂直扩展两个层面提出可扩展性评测方案,设计并实现了可扩展性评测工具——自动精简配置文件系统(thin provisioning file system,TPFS)。TPFS运行在云存储系统中的数据节点提供一定容量的实际存储空间和特定大小的元数据空间,并提供文件系统级别的自动精简配置,以此对云存储系统进行可扩展性评估。实验结果表明,在实际空间仅为40 GB的情况下,TPFS文件系统可以达到8 TB规模的扩展性。
The ability of managing large-scale data in cloud storage system is an important part of scalability of cloud storage system. However,in the evaluation of scalability,super large capacity test environment is the most difficult part. To solve this problem,this paper proposed a scalability evaluation plan in horizontal expansion and vertical expansion,designed and implemented an evaluation tool——TPFS. TPFS which ran in the data nodes of cloud storage system provide real space for data and specified space only for meta data,besides it provided thin provisioning in the level of file system,in order to evaluate the scalability of cloud storage system. The results show that when the real space is only 40 GB,TPFS file system can reach the scalability of 8 TB.
The ability of managing large-scale data in cloud storage system is an important part of scalability of cloud storage system. However,in the evaluation of scalability,super large capacity test environment is the most difficult part. To solve this problem,this paper proposed a scalability evaluation plan in horizontal expansion and vertical expansion,designed and implemented an evaluation tool——TPFS. TPFS which ran in the data nodes of cloud storage system provide real space for data and specified space only for meta data,besides it provided thin provisioning in the level of file system,in order to evaluate the scalability of cloud storage system. The results show that when the real space is only 40 GB,TPFS file system can reach the scalability of 8 TB.
引文
[1]张龙立.云存储技术探讨[J].电信科学,2010(S1):71-74.
[2]Ceph:the future of storage[EB/OL].(2015)[2015-03-12].http://ceph.com/.
[3]Gudu D,Hardt M,Streit A.Evaluating the performance and scalability of the Ceph distributed storage system[C]//Proc of IEEE International Conference on Big Data.2014:177-182.
[4]What is Apache Hadoop?[EB/OL].(2014)[2015-03-12].http://hadoop.apache.org/.
[5]Maruthachalam D,Mohanta T,Swain A R,et al.A distributed system design for next generation storage and remote replication[C]//Proc of the 5th International Conference on Applications of Digital Information and Web Technologies.2014:22-27.
[6]张冬.大话存储Ⅱ——存储系统架构与底层原理极限剖析[M].北京:清华大学出版社,2011:603-613.
[7]De Luca A,Bhide M.Storage virtualization for dummies[M].[S.l.]:Wiley Publishing Australia Pty Ltd,2009:50-53.
[8]Su Wenchi,Chang S E.Integrated cloud storage architecture for enhancing service reliability,availability and scalability[C]//Proc of International Conference on Information Science,Electronics and Electrical Engineering.2014:764-768.
[9]Mei Lijun,Chan W K,Tse T H.A tale of clouds:paradigm comparisons and some thoughts on research issues[C]//Proc of Asia-Pacific Services Computing Conference.2008:464-469.
[10]Itzhack G,Sondhi N.Storage thin provisioning and space reclamation:U.S.Patent No.9,329,792[P].2016-05-03.
[11]Kawaguchi,Tomohiro.High availability and low capacity thin provisioning:U.S.Patent Application 12/053,514[P].2008-03-21.
[12]E2fsprogs:ext2/3/4 filesystem utilities[EB/OL].(2010-10-22)[2015-03-12].http://e2fsprogs.sourceforge.net/.
[13]Master boot record[EB/OL].(2015)[2015-05-10].http://wiki.archlinux.org/index.php/Master_Boot_Record.
[14]GUID_partition_table[EB/OL].(2015)[2015-05-10].https://wiki.archlinux.org/index.php/GUID_Partition_Table.
[2]Ceph:the future of storage[EB/OL].(2015)[2015-03-12].http://ceph.com/.
[3]Gudu D,Hardt M,Streit A.Evaluating the performance and scalability of the Ceph distributed storage system[C]//Proc of IEEE International Conference on Big Data.2014:177-182.
[4]What is Apache Hadoop?[EB/OL].(2014)[2015-03-12].http://hadoop.apache.org/.
[5]Maruthachalam D,Mohanta T,Swain A R,et al.A distributed system design for next generation storage and remote replication[C]//Proc of the 5th International Conference on Applications of Digital Information and Web Technologies.2014:22-27.
[6]张冬.大话存储Ⅱ——存储系统架构与底层原理极限剖析[M].北京:清华大学出版社,2011:603-613.
[7]De Luca A,Bhide M.Storage virtualization for dummies[M].[S.l.]:Wiley Publishing Australia Pty Ltd,2009:50-53.
[8]Su Wenchi,Chang S E.Integrated cloud storage architecture for enhancing service reliability,availability and scalability[C]//Proc of International Conference on Information Science,Electronics and Electrical Engineering.2014:764-768.
[9]Mei Lijun,Chan W K,Tse T H.A tale of clouds:paradigm comparisons and some thoughts on research issues[C]//Proc of Asia-Pacific Services Computing Conference.2008:464-469.
[10]Itzhack G,Sondhi N.Storage thin provisioning and space reclamation:U.S.Patent No.9,329,792[P].2016-05-03.
[11]Kawaguchi,Tomohiro.High availability and low capacity thin provisioning:U.S.Patent Application 12/053,514[P].2008-03-21.
[12]E2fsprogs:ext2/3/4 filesystem utilities[EB/OL].(2010-10-22)[2015-03-12].http://e2fsprogs.sourceforge.net/.
[13]Master boot record[EB/OL].(2015)[2015-05-10].http://wiki.archlinux.org/index.php/Master_Boot_Record.
[14]GUID_partition_table[EB/OL].(2015)[2015-05-10].https://wiki.archlinux.org/index.php/GUID_Partition_Table.