海量数字信息存储配置的优化
|
摘要
本文探讨海量数字信息存储过程中的网络存储的配置问题,在对DAS、NAS、SAN、iSCSI-SAN等主要网络存储构架进行比较的基础上,利用排队理论对NAS、SAN和iSCSI-SAN网络存储中的排队过程进行分析和数学建模,用队列转换方法求解出SAN存储中的优化配置。
由此推导出SAN存储器级中所需存储设备的最佳数量范围,据此组成优化配置的策略,从而能帮助系统管理员更好地预先了解每种存储应用的访问模式,更合理地对存储资源进行预配置。
而iSCSI-SAN存储的存储环境较复杂,要完成该系统的配置工作相应有难度,因为其中的存储设备不再属于某个服务器,而是在存储网络中共享的,同时服务器也可以在多个存储设备上存储数据,这就需要根据用户具体的负载状况进行存储优化配置,合理分配存储资源而不造成资源的浪费。首先通过分析用户的应用负载相对存储系统的带宽比例关系,即系统负载所需的带宽与阵列所能提供的带宽的比例,来确定该存储配置是否能满足用户的需求：如果带宽比例小于等于1,表示该阵列的配置能满足用户需求；如果带宽比例大于1,表示该阵列的配置是不能满足用户需求的；如果带宽比例远小于1,表示用户的负载不能充分利用存储资源,需要根据系统负载进行重新优化配置,达到既能满足用户需求又能充分利用存储资源的目的。
根据阵列负载情况优化SAN存储配置的实例对上述结果进行了检验并对海量信息存储系统的配置可用性进行了分析。
This paper explore that the distribution of mass digital information storage in the process of network storage, first comparing several major storage architecture-AS、NAS. SAN and iSCSI-SAN, then use queuing theory to analysis the queuing process of NAS、SAN and iSCSI-SAN storage and mathematical model, using queue conversion method for solving the optimization for the distribution of the SAN.
Thus, we can conclude the optimal number range in the SAN storage arrays each class that is the optimization of the SAN storage. This would help system administrators understand the each storage in advance better, and pre-distribute the storage resources more reasonable.
The environment of the iSCSI-SAN storage is complex, so it is hard to complete the corresponding distribution of the complex systems, because the iSCSI-SAN storage device is not belong which one, but is shared in the storage network. So the optimization of the iSCSI-SAN storage distribution must be according to the user's load status, thus we can allocate the storage resources rationally without any resources wasted.
In the iSCSI-SAN storage, by analyzing the user's application of the load relative ratio between the bandwidth of storage systems, we could judge whether the storage distribution meet the needs of users:if the bandwidth ratio is less than equal to 1, indicating that the array distribution meet the user needs; if the bandwidth ratio is greater than 1, indicating that the array distribution can not meet the user needs; if the bandwidth ratio is far less than 1, indicating the user's load could not make full use of the storage resources, so,we should re-optimize the system load's distribution to meet the customer needs and make full use of the storage resources.
According the example of the optimization of the SAN storage by the array load, we have been tested the results of the above-mentioned and analyzed the availability of the distribution of the mass information storage system.
由此推导出SAN存储器级中所需存储设备的最佳数量范围,据此组成优化配置的策略,从而能帮助系统管理员更好地预先了解每种存储应用的访问模式,更合理地对存储资源进行预配置。
而iSCSI-SAN存储的存储环境较复杂,要完成该系统的配置工作相应有难度,因为其中的存储设备不再属于某个服务器,而是在存储网络中共享的,同时服务器也可以在多个存储设备上存储数据,这就需要根据用户具体的负载状况进行存储优化配置,合理分配存储资源而不造成资源的浪费。首先通过分析用户的应用负载相对存储系统的带宽比例关系,即系统负载所需的带宽与阵列所能提供的带宽的比例,来确定该存储配置是否能满足用户的需求：如果带宽比例小于等于1,表示该阵列的配置能满足用户需求；如果带宽比例大于1,表示该阵列的配置是不能满足用户需求的；如果带宽比例远小于1,表示用户的负载不能充分利用存储资源,需要根据系统负载进行重新优化配置,达到既能满足用户需求又能充分利用存储资源的目的。
根据阵列负载情况优化SAN存储配置的实例对上述结果进行了检验并对海量信息存储系统的配置可用性进行了分析。
This paper explore that the distribution of mass digital information storage in the process of network storage, first comparing several major storage architecture-AS、NAS. SAN and iSCSI-SAN, then use queuing theory to analysis the queuing process of NAS、SAN and iSCSI-SAN storage and mathematical model, using queue conversion method for solving the optimization for the distribution of the SAN.
Thus, we can conclude the optimal number range in the SAN storage arrays each class that is the optimization of the SAN storage. This would help system administrators understand the each storage in advance better, and pre-distribute the storage resources more reasonable.
The environment of the iSCSI-SAN storage is complex, so it is hard to complete the corresponding distribution of the complex systems, because the iSCSI-SAN storage device is not belong which one, but is shared in the storage network. So the optimization of the iSCSI-SAN storage distribution must be according to the user's load status, thus we can allocate the storage resources rationally without any resources wasted.
In the iSCSI-SAN storage, by analyzing the user's application of the load relative ratio between the bandwidth of storage systems, we could judge whether the storage distribution meet the needs of users:if the bandwidth ratio is less than equal to 1, indicating that the array distribution meet the user needs; if the bandwidth ratio is greater than 1, indicating that the array distribution can not meet the user needs; if the bandwidth ratio is far less than 1, indicating the user's load could not make full use of the storage resources, so,we should re-optimize the system load's distribution to meet the customer needs and make full use of the storage resources.
According the example of the optimization of the SAN storage by the array load, we have been tested the results of the above-mentioned and analyzed the availability of the distribution of the mass information storage system.
引文
[1]张江陵,冯丹等.海量信息存储[M].北京：科学出版社.2003
[2]Seagate technology. Network attached atorage&Object oriented devices. 2005.http://www. Seagate. com/support/kb/disc/nas_ood_env_custers. html
[3]Gobioff, H., Nagle, D. F. and Gibson, G. A.. Embedded security for network attached storage. CMU SCS Technical Report CMU-CS-99-154,June 1999
[4]Gobioff H., Nagle D. F. and Gibson, G. A.. Integrity and performance in network attached storage. CMU SCS Technical Report CMU-CS-98-182, December 1998
[5]Yiming Hu,Qing Yang. DCD-Disk caching disk. A new approach for boosting I/O performance. Proceedings of ISCA96 [J],Philadelphia, PA,1996:169-177
[6]Yiming Hu,Qing Yang. A new hierarchical disk architecture [J],IEEE Micro,1998,18 (6):64-76
[7]Tycho Nightingale, Yiming Hu, Qing Yang. The design and implementation of a DCD device driver for Unix. In:Proceedings of the 1999 USENIX Annual Technical Conference, USA:USENIX,1999:295-307
[8]Qing Yang, Yiming Hu. Disk Caching Disk-A new device for high perfor-mance I/O system[J]. U.S. Patent and Trademark Off ice,No.5754,888. September 24th,1997
[9]Xubin He, Qing Yang, and Ming Zhang. A caching strategy to improve iSCSI performance[J]. Proceedings of LCN2002 USA IEEE Computer Society, 2002:278-288
[10]Xubin He, Qing Yang, and Ming Zhang. Introducing SCSI-to-IP cache for storage area networks [J]. Proceedings of ICPP'2002 USA, IEEE Computer Society,2002:203-210
[11]Xubin He, Ming Zhang, Qing Yang. SCSI-to-IP cache for storage area networks[J]. Journal of Parallel and Distributed Computing,2004, 64(9):1069-1085
[12]Garth A., Gibson. Network-attached storage architecture[J]. Communication of the ACM,2000,43(2):11-17
[13]DAFS Collaborative. Direct access file system protocol, Version 1.0. September 2001. http://www.dafscollaborative.org
[14]J. Griffin. Operating system management of MEMS-based storage devices. 4th Symp. on Operating systems design and implementation (OSDI). San Diego, California:Usenix Association, Oct 2000:227-242
[15]Fred Douglis. Storage alternatives for mobile computers. Proceedings of the 1st Symposium on Operating Systems Design and Implementation Monterey, CA:USENIX, November 1994:25-37
[16]M. Wu, W. Zwaenepoel. eNVy:a non-volatile, main memory storage system".6th Conf. On Architectural Support for Programming Languages and Operating Systems. San Jose, CA, October 1994:86-97
[17]R. English,A. Stepanov. Loge:A self-organizing disk controller. Winter 1992 USENIX Technical Conference. San Francisco, CA, 1992:237-252
[18]Vivek S. Pai, Peter Druschel,Willy Zwaenepoel. IO-Lite:A unified I/O buffering and caching system. ACM Transaction on Computer Systems, 2000,18(1):37-66
[19]Lumb C.,Schindler J., Ganger G., Riedel. Towards higher disk head utilization extracting "free" bandwidth from busy disk drives. Proc. of the Fourth Symposium on Operating Systems Design and Implementation. San Diego, CA, October 2000:87-102
[20]J.Wilkes, R. Golding, C.Staelin. The HP auto RAID hierarchical storage system. ACM Transactions on Computer Systems,1996,14 (1):108-136
[21]Venkelesh Pisillil. Design implementation and policy framework for a Linux based temperature sensitive storage. Proceedings of 5th Annual Linux Showcase & Conference. Oakland:USENIX Press,2001: 107-118
[22]R. J. Honicky, Ethan L. Miller. An optimal algorithm for online reorganization of replicated data. Technical Report UCSC-CRL-02-36. Storage Systems Research Center, University of California, Santa Cruz, November 2002
[23]R. J Honicky, E. L. Miller. A fast algorithm for online placement and reorganization of replicated data.Proceedings of the 17th International Parallel & Distributed Processing Symposium. Nice, France, Apr 2003:57-68
[24]Edward K. Lee, Chandramohan A. Thekkath. Petal:Distributed virtual disk. Proceedings of 7th Intl. conf. on Architectural Support for Programming Languages and Operation Systems. Cambridge, MA, October 1996:84-92
[25]David Teigland, Heinz Mauelshagen. Volume managers in Linux in proceedings of the 2001 USENIX Annual Techinical Conference, pp. 185-198, June 2001
[26]Shu Jiwu, Yu Bing, Yan Rui. A design and implementation of non-volatile RAM disk in SAN environment. In:H. Jin, Y Pan, N.Xiao and J. Sun eds.Proceedings of GCC’ 2004Workshop on Storage Grid and Technologies,Germany:Springer,2004:203-212
[27]刘朝斌.虚拟网络存储系统关键技术研究及其性能评价[博士论文].武汉：华中科技大学,2004
[28]谢长生,高巍.存储区域网(SAN)中存储虚拟化的研究与实现[J].计算机应用研究,2003,20(8)：130-132
[29]付长冬,舒继武,沈美明,郑纬民.基于RAM／DISK混合设备模型的FC-SAN存储系统[J].计算机研究与发展,2004,41(3)：504-511
[30]付长冬,舒继武,沈美明,郑纬民.网络存储体系结构的研究和发展[J].小型 微型计算机系统,2004,25(4)：485-489
[31]付长冬,舒继武,沈美明,郑纬民.一个高性能的FC-SAN存储系统的设计与实现[J].高技术通讯,2003,13(154)：25-29
[32]胡长军,田金兰,舒继武.网络存储系统体系结构研究综述.CNCC 03(2003中国计算机大会)论文集,中国：清华大学出版社,2003：1163-1168
[33]韩德志,谢长生,傅湘林,易法令.一种基于iSCSI的附网存储服务器系统的设计与实现[J].计算机研究与发展,2004,41(1)：207-213
[34]谢长生,傅湘林,韩德志,任劲.一种基于iSCSI的SAN的研究与实现[J].计算机研究与发展,2003,40(5)：747-751
[35]周敬利,张威,余胜生.iSCSI SAN中存储管理器的设计与实现[J].计算机工程与应用,2004,12：96-118
[36]冯丹等.基于多通道存储设备的海量存储系统及其系统控制方法.中国,发明专利,ZL03125249.4,2005-11-23
[37]张江陵等.外置式多通道网络磁盘阵列控制装置及其使用的协议适配方法.中国,发明专利,ZL03125247.8,2005-09-28
[38]冯泳.SAN存储资源管理系统中性能监视及优化技术的研究与实现.[硕士学位论文].中国：西北工业大学,2002
[39]Luyp, David. Performance study of iSCSI-based storage subsystems [J]. IEEE Communications Magazine,2003,41 (8):76-82
[40]Hexb, Beedanagari, Zhou D. Performance evaluation of distributed iSCSI RAID. Proceedings of the 2003 IEEE/ACM International Workshop on Storage Network Architecture and Parallel I/O (SNAP’ 03). New Orleans,LA, USA,2003
[41]Weetn, Hillyer, Shriver E. Obtaining high performance for storage outsourcing. Procee-dings of Conference on File and Storage Technolo-gies (FAST’02). Monterey, California,2002:145-158
[42]曹强,谢长生.网络存储系统中I／O请求时间的研究[J].计算机研究与发展,2003,40(8)：1271-1275
[43]崔宝江等.网络存储系统I／O响应时间边界性能研究[J].通信学报,2006, 27(1)：70-74
[44]周薇等.排队论在计算机存储系统性能中的应用和分析[J].微计算机信息,2006.22(7-3)：271-296
[45]董臻圃等.数学建模方法与实践[M].北京：国防工业出版社,2006
[46]胡运权主编.运筹学教程[M].北京：清华大学,2007：306-338
[47]罗登文.海量存储系统设备和模式的选择[J].现代情报.2004(11)：98
[48]周敬利等.网络存储原理与技术[M].北京：清华大学出版社.2005
[49]牛奎光等.轻型海量信息存储系统的设计和实现[J].计算机工程.2005(5)：72
[50]解读网络存储技术iSCSI.天极网.http://www. chinabyte. net/20021031/1637617. shtml
[51]李波,孙美娜.数字图书馆海量存储的理想选择—IP-SAN[J].科技情报开发与经济.2007(4)
[52]谢长生,傅湘林,韩德志,任劲.一种基于iSCSI的SAN的研究与实现[J].计算机研究与发展,2003,40(5)：747-751
[53]师鸣若,王改性主编.数据存储备份与灾难恢复[M].北京：电子工业出版社,2009
[54]Ruemmler C.,Wilkes John. An introduction to disk drive modeling[J]. IEEE Computer,Mar1994,27(3):17-20
[55]张江陵.高性能磁盘阵列I／O服务时间分析[J].小型微型计算机系统,2000,21(3)：235-237
[56]向东.iSCSI-SAN网络异构存储系统管理策略的研究.[博士学位论文].中国：华中科技大学,2002
[57]崔梦天等.基于排队论的方法解决分布式系统中信息资源的问题[J].情报杂志,2005(3)：46-47
[58]Tools development staff.Iometer user guide. Intel Corporation,2001. URL:http://sourceforge. net/projects/iometer/
[59]C. Ruemmler, J. Wilkes. Unix disk access patterns. Winter USENIX, January 1993:405-420
[60]傅湘林.统一存储网络关键技术研究.武汉：华中科技大学图书馆,2004
[61]林闯.计算机网络和计算机系统的性能评价(第1版)[M].北京：清华大学出版社,2001：356
[2]Seagate technology. Network attached atorage&Object oriented devices. 2005.http://www. Seagate. com/support/kb/disc/nas_ood_env_custers. html
[3]Gobioff, H., Nagle, D. F. and Gibson, G. A.. Embedded security for network attached storage. CMU SCS Technical Report CMU-CS-99-154,June 1999
[4]Gobioff H., Nagle D. F. and Gibson, G. A.. Integrity and performance in network attached storage. CMU SCS Technical Report CMU-CS-98-182, December 1998
[5]Yiming Hu,Qing Yang. DCD-Disk caching disk. A new approach for boosting I/O performance. Proceedings of ISCA96 [J],Philadelphia, PA,1996:169-177
[6]Yiming Hu,Qing Yang. A new hierarchical disk architecture [J],IEEE Micro,1998,18 (6):64-76
[7]Tycho Nightingale, Yiming Hu, Qing Yang. The design and implementation of a DCD device driver for Unix. In:Proceedings of the 1999 USENIX Annual Technical Conference, USA:USENIX,1999:295-307
[8]Qing Yang, Yiming Hu. Disk Caching Disk-A new device for high perfor-mance I/O system[J]. U.S. Patent and Trademark Off ice,No.5754,888. September 24th,1997
[9]Xubin He, Qing Yang, and Ming Zhang. A caching strategy to improve iSCSI performance[J]. Proceedings of LCN2002 USA IEEE Computer Society, 2002:278-288
[10]Xubin He, Qing Yang, and Ming Zhang. Introducing SCSI-to-IP cache for storage area networks [J]. Proceedings of ICPP'2002 USA, IEEE Computer Society,2002:203-210
[11]Xubin He, Ming Zhang, Qing Yang. SCSI-to-IP cache for storage area networks[J]. Journal of Parallel and Distributed Computing,2004, 64(9):1069-1085
[12]Garth A., Gibson. Network-attached storage architecture[J]. Communication of the ACM,2000,43(2):11-17
[13]DAFS Collaborative. Direct access file system protocol, Version 1.0. September 2001. http://www.dafscollaborative.org
[14]J. Griffin. Operating system management of MEMS-based storage devices. 4th Symp. on Operating systems design and implementation (OSDI). San Diego, California:Usenix Association, Oct 2000:227-242
[15]Fred Douglis. Storage alternatives for mobile computers. Proceedings of the 1st Symposium on Operating Systems Design and Implementation Monterey, CA:USENIX, November 1994:25-37
[16]M. Wu, W. Zwaenepoel. eNVy:a non-volatile, main memory storage system".6th Conf. On Architectural Support for Programming Languages and Operating Systems. San Jose, CA, October 1994:86-97
[17]R. English,A. Stepanov. Loge:A self-organizing disk controller. Winter 1992 USENIX Technical Conference. San Francisco, CA, 1992:237-252
[18]Vivek S. Pai, Peter Druschel,Willy Zwaenepoel. IO-Lite:A unified I/O buffering and caching system. ACM Transaction on Computer Systems, 2000,18(1):37-66
[19]Lumb C.,Schindler J., Ganger G., Riedel. Towards higher disk head utilization extracting "free" bandwidth from busy disk drives. Proc. of the Fourth Symposium on Operating Systems Design and Implementation. San Diego, CA, October 2000:87-102
[20]J.Wilkes, R. Golding, C.Staelin. The HP auto RAID hierarchical storage system. ACM Transactions on Computer Systems,1996,14 (1):108-136
[21]Venkelesh Pisillil. Design implementation and policy framework for a Linux based temperature sensitive storage. Proceedings of 5th Annual Linux Showcase & Conference. Oakland:USENIX Press,2001: 107-118
[22]R. J. Honicky, Ethan L. Miller. An optimal algorithm for online reorganization of replicated data. Technical Report UCSC-CRL-02-36. Storage Systems Research Center, University of California, Santa Cruz, November 2002
[23]R. J Honicky, E. L. Miller. A fast algorithm for online placement and reorganization of replicated data.Proceedings of the 17th International Parallel & Distributed Processing Symposium. Nice, France, Apr 2003:57-68
[24]Edward K. Lee, Chandramohan A. Thekkath. Petal:Distributed virtual disk. Proceedings of 7th Intl. conf. on Architectural Support for Programming Languages and Operation Systems. Cambridge, MA, October 1996:84-92
[25]David Teigland, Heinz Mauelshagen. Volume managers in Linux in proceedings of the 2001 USENIX Annual Techinical Conference, pp. 185-198, June 2001
[26]Shu Jiwu, Yu Bing, Yan Rui. A design and implementation of non-volatile RAM disk in SAN environment. In:H. Jin, Y Pan, N.Xiao and J. Sun eds.Proceedings of GCC’ 2004Workshop on Storage Grid and Technologies,Germany:Springer,2004:203-212
[27]刘朝斌.虚拟网络存储系统关键技术研究及其性能评价[博士论文].武汉：华中科技大学,2004
[28]谢长生,高巍.存储区域网(SAN)中存储虚拟化的研究与实现[J].计算机应用研究,2003,20(8)：130-132
[29]付长冬,舒继武,沈美明,郑纬民.基于RAM／DISK混合设备模型的FC-SAN存储系统[J].计算机研究与发展,2004,41(3)：504-511
[30]付长冬,舒继武,沈美明,郑纬民.网络存储体系结构的研究和发展[J].小型 微型计算机系统,2004,25(4)：485-489
[31]付长冬,舒继武,沈美明,郑纬民.一个高性能的FC-SAN存储系统的设计与实现[J].高技术通讯,2003,13(154)：25-29
[32]胡长军,田金兰,舒继武.网络存储系统体系结构研究综述.CNCC 03(2003中国计算机大会)论文集,中国：清华大学出版社,2003：1163-1168
[33]韩德志,谢长生,傅湘林,易法令.一种基于iSCSI的附网存储服务器系统的设计与实现[J].计算机研究与发展,2004,41(1)：207-213
[34]谢长生,傅湘林,韩德志,任劲.一种基于iSCSI的SAN的研究与实现[J].计算机研究与发展,2003,40(5)：747-751
[35]周敬利,张威,余胜生.iSCSI SAN中存储管理器的设计与实现[J].计算机工程与应用,2004,12：96-118
[36]冯丹等.基于多通道存储设备的海量存储系统及其系统控制方法.中国,发明专利,ZL03125249.4,2005-11-23
[37]张江陵等.外置式多通道网络磁盘阵列控制装置及其使用的协议适配方法.中国,发明专利,ZL03125247.8,2005-09-28
[38]冯泳.SAN存储资源管理系统中性能监视及优化技术的研究与实现.[硕士学位论文].中国：西北工业大学,2002
[39]Luyp, David. Performance study of iSCSI-based storage subsystems [J]. IEEE Communications Magazine,2003,41 (8):76-82
[40]Hexb, Beedanagari, Zhou D. Performance evaluation of distributed iSCSI RAID. Proceedings of the 2003 IEEE/ACM International Workshop on Storage Network Architecture and Parallel I/O (SNAP’ 03). New Orleans,LA, USA,2003
[41]Weetn, Hillyer, Shriver E. Obtaining high performance for storage outsourcing. Procee-dings of Conference on File and Storage Technolo-gies (FAST’02). Monterey, California,2002:145-158
[42]曹强,谢长生.网络存储系统中I／O请求时间的研究[J].计算机研究与发展,2003,40(8)：1271-1275
[43]崔宝江等.网络存储系统I／O响应时间边界性能研究[J].通信学报,2006, 27(1)：70-74
[44]周薇等.排队论在计算机存储系统性能中的应用和分析[J].微计算机信息,2006.22(7-3)：271-296
[45]董臻圃等.数学建模方法与实践[M].北京：国防工业出版社,2006
[46]胡运权主编.运筹学教程[M].北京：清华大学,2007：306-338
[47]罗登文.海量存储系统设备和模式的选择[J].现代情报.2004(11)：98
[48]周敬利等.网络存储原理与技术[M].北京：清华大学出版社.2005
[49]牛奎光等.轻型海量信息存储系统的设计和实现[J].计算机工程.2005(5)：72
[50]解读网络存储技术iSCSI.天极网.http://www. chinabyte. net/20021031/1637617. shtml
[51]李波,孙美娜.数字图书馆海量存储的理想选择—IP-SAN[J].科技情报开发与经济.2007(4)
[52]谢长生,傅湘林,韩德志,任劲.一种基于iSCSI的SAN的研究与实现[J].计算机研究与发展,2003,40(5)：747-751
[53]师鸣若,王改性主编.数据存储备份与灾难恢复[M].北京：电子工业出版社,2009
[54]Ruemmler C.,Wilkes John. An introduction to disk drive modeling[J]. IEEE Computer,Mar1994,27(3):17-20
[55]张江陵.高性能磁盘阵列I／O服务时间分析[J].小型微型计算机系统,2000,21(3)：235-237
[56]向东.iSCSI-SAN网络异构存储系统管理策略的研究.[博士学位论文].中国：华中科技大学,2002
[57]崔梦天等.基于排队论的方法解决分布式系统中信息资源的问题[J].情报杂志,2005(3)：46-47
[58]Tools development staff.Iometer user guide. Intel Corporation,2001. URL:http://sourceforge. net/projects/iometer/
[59]C. Ruemmler, J. Wilkes. Unix disk access patterns. Winter USENIX, January 1993:405-420
[60]傅湘林.统一存储网络关键技术研究.武汉：华中科技大学图书馆,2004
[61]林闯.计算机网络和计算机系统的性能评价(第1版)[M].北京：清华大学出版社,2001：356