树型结构存储系统设计与层间缓存技术
|
摘要
磁盘阵列在海量存储系统中有着广泛的应用,然而其系统总线带宽限制了底层通道的并行性,存储系统内部接口性能的差异制约着存储系统性能的提升。因此,消除系统总线带宽瓶颈和接口性能差异对提升存储系统整体性能具有重要的研究意义。
树型结构存储系统采用树型结构对数据进行组织和管理,改变了磁盘阵列传统的数据组织管理模式。树型结构多通道、高并行的特点能有效解决存储系统内部的总线带宽瓶颈,降低总线争用概率,提高总线利用率;树型结构的高扩展性增强了存储系统的可扩展性和兼容性;树型结构存储系统利用高速存储设备对低速通道连接的存储设备上的数据进行缓存,实现层间缓存技术,减小低速通道对系统性能的制约,有效提高了系统的整体性能。
单元控制器是构成树型结构存储系统的主要部件,通过上行高速通道和下行低速通道连接到主机或其它单元控制器,磁盘构成系统的叶子节点。树结构存储系统提出了三种数据分块方式:无校验块交叉方式(DT0),分级镜像方式(DT1),分级奇偶校验方式(DT2)。树型结构存储系统有效解决了总线带宽瓶颈、通道带宽匹配、集合总线带宽等问题,并使用不同接口标准的存储设备进行并行数据传输。与磁盘阵列相比,树型结构存储系统提高了存储系统的并行性与扩展性。
Redundant arrays of inexpensive disks are widely used in storage servers. But the finite bandwidth of the system bus limits the parallel of the bottom storage channels; Performance of each access technologies used in the storage system also limits the performance of system. So it is necessary to solve the bottleneck of the system bus and lessen the performance distinction of the access technologies.
Tree structure storage system uses the tree structure for the data organization and management, and changed the tranditional model of data management. Tree structure can solve the bottleneck of the system bus and reduce the competition probability effectively, because of its multi-channel character and its parallel; The high scalability of the tree structure make the storage system update easily, and also enhance the scalability of the storage system; cache technology of the channel that is used in the tree structure storage system can lessen the performance distinction of the access technologies.
Tree structure storage system is consisted of control cell, that use upward channel and downtrend channel to connect to the host or other control cell. The leafage of this storage system is made of disks. There are three main method for the data distribution of the tree structure storage system: striping method(DT0), Mirroring distribution method(DT1), Parity distribution method(DT2).Tree structure have solved the problems of bandwidth matching, polymerization bandwidth and so on. It makes the devices that have different access technology work together and transfer the data parallel. Compared with RAID, tree structure storage system increases the parallel and expansibility of the storage system.
树型结构存储系统采用树型结构对数据进行组织和管理,改变了磁盘阵列传统的数据组织管理模式。树型结构多通道、高并行的特点能有效解决存储系统内部的总线带宽瓶颈,降低总线争用概率,提高总线利用率;树型结构的高扩展性增强了存储系统的可扩展性和兼容性;树型结构存储系统利用高速存储设备对低速通道连接的存储设备上的数据进行缓存,实现层间缓存技术,减小低速通道对系统性能的制约,有效提高了系统的整体性能。
单元控制器是构成树型结构存储系统的主要部件,通过上行高速通道和下行低速通道连接到主机或其它单元控制器,磁盘构成系统的叶子节点。树结构存储系统提出了三种数据分块方式:无校验块交叉方式(DT0),分级镜像方式(DT1),分级奇偶校验方式(DT2)。树型结构存储系统有效解决了总线带宽瓶颈、通道带宽匹配、集合总线带宽等问题,并使用不同接口标准的存储设备进行并行数据传输。与磁盘阵列相比,树型结构存储系统提高了存储系统的并行性与扩展性。
Redundant arrays of inexpensive disks are widely used in storage servers. But the finite bandwidth of the system bus limits the parallel of the bottom storage channels; Performance of each access technologies used in the storage system also limits the performance of system. So it is necessary to solve the bottleneck of the system bus and lessen the performance distinction of the access technologies.
Tree structure storage system uses the tree structure for the data organization and management, and changed the tranditional model of data management. Tree structure can solve the bottleneck of the system bus and reduce the competition probability effectively, because of its multi-channel character and its parallel; The high scalability of the tree structure make the storage system update easily, and also enhance the scalability of the storage system; cache technology of the channel that is used in the tree structure storage system can lessen the performance distinction of the access technologies.
Tree structure storage system is consisted of control cell, that use upward channel and downtrend channel to connect to the host or other control cell. The leafage of this storage system is made of disks. There are three main method for the data distribution of the tree structure storage system: striping method(DT0), Mirroring distribution method(DT1), Parity distribution method(DT2).Tree structure have solved the problems of bandwidth matching, polymerization bandwidth and so on. It makes the devices that have different access technology work together and transfer the data parallel. Compared with RAID, tree structure storage system increases the parallel and expansibility of the storage system.
引文
[1]张悠慧,郑纬民.一种新的网络对象存储设备研究.电子学报, 2003, 31(5): 679~682
[2]张江陵,冯丹.海量信息存储(第一版).北京:科学出版社, 2003. 85~90
[3]郑纬民,舒继武.下一代分布式智能网络存储系统的发展趋势.世界电信, 2004, 17(8):1~2
[4] Gilbson.G, Meter.V.R. Network Attached Storage Architecture. Communications of the ACM, 2000, 43(11):37~45
[5] Seigle.M. Storage area networks in video applications. SMPTE Journal, 2001, 110(4):236~239
[6]张羚.基于对象存储存储控制器的软件设计及实现:[硕士学位论文].武汉:华中科技大学图书馆, 2006.
[7] Patterson.D, Gibson.G, Katz.R. A Case for Redundant Arrays of Inexpensive Disks(RAID). In: Proceedings of ACM SIGMOD Conference on the Management of Data. New York, USA:ACM Conference Press, 1988. 109~116
[8] David.K.G. Poly channel systems for mass digital communications. Communi-cations of the ACM, 1990, 33(2):141~151
[9] Chen.P, Lee.E.K, Gibson.G et al. RAID: High-Performance, Reliable Secondary Storage.ACM Computing Surveys, 1994, 26(2):145~185
[10]顾锦旗,朱平.一种新型外存储磁盘阵列的设计.电子计算机与外部设备, 1998, 22(3):12~16
[11] Wilkes.J, Golding.R. The HP AutoRAID Hierarchical Storage System.ACM Transfer On Computer Systems, 1996, 14(1):108~136
[12] Colarelli.D, Grunwald.D. Massive Arrays of Idle Disks For Storage Archives. In: Proceedings of the 15th High Performance Networking and Computing Conference. Baltimore, USA: IEEE Computer Society Press, 2002. 1~11
[13]孟庆飞.网络磁盘阵列系统研究:[硕士学位论文].武汉:华中科技大学图书馆, 2001.
[14]王芳.网络磁盘阵列系统的研究:[博士学位论文].武汉:华中科技大学图书馆, 2002.
[15]周可.外存储系统数据组织与体系结构:[博士学位论文].武汉:华中科技大学图书馆, 2003.
[16] Ke Zhou, Yongfeng Huang, Dan Feng. Disk Tree—A Case of Parallel Storage Architecture to improve performance in random access pattern. Chinese Journal of Electronics, 2005, 14(1):39~44
[17] Nelson.J.M, LeVine.M.J, Ljubicic.A.Jr, et al. Inter processor data via SCSI-2. In: Real Time Conference, 1999, Santa Fe 1999.11th IEEE NPSS. Sante, USA: IEEE Computer Society Press, 1999. 422~423
[18]谢希仁.计算机网络(第三版).大连:大连理工大学出版社, 2006. 68~69
[19] Leiserson.C.E. Fat-trees: Universal networks for hardware-efficient supercomputing. IEEE Transactions on Computers, 1985, 34(10): 892~901
[20]黄志钧.分层存储要慎重.网管员世界, 2006, 4(6):5~6
[21] Jian.X.C, Dan.F, Zhan.S. iVISA: A Framework for Flexible Layout Block-level Storage System. In:Proceedings of the 20th International Conference on Advanced Information Networking and Applications(AINA’06), 2006, 4(2):5~10
[22] Cornelius.R.S. Overview and status of Fibre Channel. Compcon Spring '94, 1994, 23(3):74~77
[23] Frymoyer.E.M. Fibre Channel application profiles, FCSI and FCA. In: Compcon Spring’94, Digest of Papers. San Francisco, USA: IEEE Conference Press, 1994. 83~92
[24]叶敏.基于VIA的光纤通道交换网MPI的研究与实现:[硕士学位论文].西安:西北工业大学图书馆, 2005.
[25]童薇,冯丹,张江陵.磁盘阵列中的聚散技术研究.计算机工程, 2001, 29(3):45~49
[26]王芳,冯丹,张江陵. RAID中基于聚/散的命令合并的相关问题研究.微电子学与计算机, 2003, 8(6):27~29
[27]陈安定,谢汶. Linux下的逻辑卷管理.四川电力技术, 2003, 3(1):47~48
[28]胡晓晖,刘桂喜. AIX逻辑卷管理技术.现代计算机, 2001, 7(6):96~98
[29] Gao.K, Pei.J, Zheng.H.X et al. Implementation of EIDE disk array system for mass data backup. In: Aerospace and Electronic Systems Magazine IEEE, 2004, 19(11):24~29
[30] Qlogic Corporation. ISP2300/2310/2312 Intelligent SCSI Processors Fireware Interface Specification. Qlogic Corporation, 2000. 1~8
[31] Namgoong.J.C, Park.C. Design and Implementation of a Fibre Channel Network Driver for SAN-attached RAID Controller. In: Eight International Conference onParallel and Distributed Systems. Kyongju, China: IEEE Conference Press, 2001. 477~483
[32]田磊.光纤卡阵列中逻辑卷管理与光纤通道接口研究:[硕士学位论文].武汉:华中科技大学图书馆, 2004.
[33]余胜生,王国华,周敬利. Linux下iSCSI存储系统中逻辑卷管理.计算机工程, 2003, 30(11):29~30
[34] Wang.P, Gilligan.E.R, Green.H et al. IP SAN-From iSCSI to IP-addressable Ethernet Disks. In: Proceedings of the 20th IEEE/11 the NASA Goddard Conference on Mass Storage Systems and Technologies(MSS’03). San Diego, USA: IEEE Conference Press, 2003. 189~193
[35] Meth.K.Z, Satran.J. Features of the iSCSI protocol. Communications Magazine, IEEE, 2003, 41(8):72~75
[36] Ke Zhou, Jiangling Zhang, Dan Feng et al. Cache Prefetching Adaptive Policy Based on Access Pattern. In: Proceedings of the First International Conference on Machine Learning and Cybernetics. Beijing, China: IEEE Press, 2002. 496~500
[37] Corbet.J, Rubini.A, Hartman.K.G. Linux设备驱动程序(第三版).魏永明等译.北京:中国电力出版社, 2006. 458~487
[38]邓玉辉.基于网络磁盘阵列的海量信息存储系统:[博士学位论文].武汉:华中科技大学图书馆, 2004.
[39] He.X.B, Yang.Q, Zhang.M. A caching strategy to improve iSCSI performance. In: Proceedings of the 27th Annual IEEE Conference on Local Computer Networks (LCN’02). Dublin, Ireland: IEEE Conference Press, 2002. 278~285
[40] Hu.Y, Yang.Q. DCD-Disk Caching Disk: A New Approach for Boosting I/O Performance. In: The 23rd International Symposium on Computer Architecture. Philadelphia, USA: IEEE Computer Society Press, 1996. 169~178
[41]姜戬,谢长生,贺军. Windows NT环境下缓存磁盘技术驱动程序的设计与实现.计算机工程与科学, 2001, 23(2):90~93
[42]李洁琼.基于网络磁盘阵列的海量信息存储系统研究:[硕士学位论文].武汉:华中科技大学图书馆, 2003.
[43]冯丹.外存储系统并行性研究. [博士学位论文].武汉:华中科技大学图书馆, 1997.
[2]张江陵,冯丹.海量信息存储(第一版).北京:科学出版社, 2003. 85~90
[3]郑纬民,舒继武.下一代分布式智能网络存储系统的发展趋势.世界电信, 2004, 17(8):1~2
[4] Gilbson.G, Meter.V.R. Network Attached Storage Architecture. Communications of the ACM, 2000, 43(11):37~45
[5] Seigle.M. Storage area networks in video applications. SMPTE Journal, 2001, 110(4):236~239
[6]张羚.基于对象存储存储控制器的软件设计及实现:[硕士学位论文].武汉:华中科技大学图书馆, 2006.
[7] Patterson.D, Gibson.G, Katz.R. A Case for Redundant Arrays of Inexpensive Disks(RAID). In: Proceedings of ACM SIGMOD Conference on the Management of Data. New York, USA:ACM Conference Press, 1988. 109~116
[8] David.K.G. Poly channel systems for mass digital communications. Communi-cations of the ACM, 1990, 33(2):141~151
[9] Chen.P, Lee.E.K, Gibson.G et al. RAID: High-Performance, Reliable Secondary Storage.ACM Computing Surveys, 1994, 26(2):145~185
[10]顾锦旗,朱平.一种新型外存储磁盘阵列的设计.电子计算机与外部设备, 1998, 22(3):12~16
[11] Wilkes.J, Golding.R. The HP AutoRAID Hierarchical Storage System.ACM Transfer On Computer Systems, 1996, 14(1):108~136
[12] Colarelli.D, Grunwald.D. Massive Arrays of Idle Disks For Storage Archives. In: Proceedings of the 15th High Performance Networking and Computing Conference. Baltimore, USA: IEEE Computer Society Press, 2002. 1~11
[13]孟庆飞.网络磁盘阵列系统研究:[硕士学位论文].武汉:华中科技大学图书馆, 2001.
[14]王芳.网络磁盘阵列系统的研究:[博士学位论文].武汉:华中科技大学图书馆, 2002.
[15]周可.外存储系统数据组织与体系结构:[博士学位论文].武汉:华中科技大学图书馆, 2003.
[16] Ke Zhou, Yongfeng Huang, Dan Feng. Disk Tree—A Case of Parallel Storage Architecture to improve performance in random access pattern. Chinese Journal of Electronics, 2005, 14(1):39~44
[17] Nelson.J.M, LeVine.M.J, Ljubicic.A.Jr, et al. Inter processor data via SCSI-2. In: Real Time Conference, 1999, Santa Fe 1999.11th IEEE NPSS. Sante, USA: IEEE Computer Society Press, 1999. 422~423
[18]谢希仁.计算机网络(第三版).大连:大连理工大学出版社, 2006. 68~69
[19] Leiserson.C.E. Fat-trees: Universal networks for hardware-efficient supercomputing. IEEE Transactions on Computers, 1985, 34(10): 892~901
[20]黄志钧.分层存储要慎重.网管员世界, 2006, 4(6):5~6
[21] Jian.X.C, Dan.F, Zhan.S. iVISA: A Framework for Flexible Layout Block-level Storage System. In:Proceedings of the 20th International Conference on Advanced Information Networking and Applications(AINA’06), 2006, 4(2):5~10
[22] Cornelius.R.S. Overview and status of Fibre Channel. Compcon Spring '94, 1994, 23(3):74~77
[23] Frymoyer.E.M. Fibre Channel application profiles, FCSI and FCA. In: Compcon Spring’94, Digest of Papers. San Francisco, USA: IEEE Conference Press, 1994. 83~92
[24]叶敏.基于VIA的光纤通道交换网MPI的研究与实现:[硕士学位论文].西安:西北工业大学图书馆, 2005.
[25]童薇,冯丹,张江陵.磁盘阵列中的聚散技术研究.计算机工程, 2001, 29(3):45~49
[26]王芳,冯丹,张江陵. RAID中基于聚/散的命令合并的相关问题研究.微电子学与计算机, 2003, 8(6):27~29
[27]陈安定,谢汶. Linux下的逻辑卷管理.四川电力技术, 2003, 3(1):47~48
[28]胡晓晖,刘桂喜. AIX逻辑卷管理技术.现代计算机, 2001, 7(6):96~98
[29] Gao.K, Pei.J, Zheng.H.X et al. Implementation of EIDE disk array system for mass data backup. In: Aerospace and Electronic Systems Magazine IEEE, 2004, 19(11):24~29
[30] Qlogic Corporation. ISP2300/2310/2312 Intelligent SCSI Processors Fireware Interface Specification. Qlogic Corporation, 2000. 1~8
[31] Namgoong.J.C, Park.C. Design and Implementation of a Fibre Channel Network Driver for SAN-attached RAID Controller. In: Eight International Conference onParallel and Distributed Systems. Kyongju, China: IEEE Conference Press, 2001. 477~483
[32]田磊.光纤卡阵列中逻辑卷管理与光纤通道接口研究:[硕士学位论文].武汉:华中科技大学图书馆, 2004.
[33]余胜生,王国华,周敬利. Linux下iSCSI存储系统中逻辑卷管理.计算机工程, 2003, 30(11):29~30
[34] Wang.P, Gilligan.E.R, Green.H et al. IP SAN-From iSCSI to IP-addressable Ethernet Disks. In: Proceedings of the 20th IEEE/11 the NASA Goddard Conference on Mass Storage Systems and Technologies(MSS’03). San Diego, USA: IEEE Conference Press, 2003. 189~193
[35] Meth.K.Z, Satran.J. Features of the iSCSI protocol. Communications Magazine, IEEE, 2003, 41(8):72~75
[36] Ke Zhou, Jiangling Zhang, Dan Feng et al. Cache Prefetching Adaptive Policy Based on Access Pattern. In: Proceedings of the First International Conference on Machine Learning and Cybernetics. Beijing, China: IEEE Press, 2002. 496~500
[37] Corbet.J, Rubini.A, Hartman.K.G. Linux设备驱动程序(第三版).魏永明等译.北京:中国电力出版社, 2006. 458~487
[38]邓玉辉.基于网络磁盘阵列的海量信息存储系统:[博士学位论文].武汉:华中科技大学图书馆, 2004.
[39] He.X.B, Yang.Q, Zhang.M. A caching strategy to improve iSCSI performance. In: Proceedings of the 27th Annual IEEE Conference on Local Computer Networks (LCN’02). Dublin, Ireland: IEEE Conference Press, 2002. 278~285
[40] Hu.Y, Yang.Q. DCD-Disk Caching Disk: A New Approach for Boosting I/O Performance. In: The 23rd International Symposium on Computer Architecture. Philadelphia, USA: IEEE Computer Society Press, 1996. 169~178
[41]姜戬,谢长生,贺军. Windows NT环境下缓存磁盘技术驱动程序的设计与实现.计算机工程与科学, 2001, 23(2):90~93
[42]李洁琼.基于网络磁盘阵列的海量信息存储系统研究:[硕士学位论文].武汉:华中科技大学图书馆, 2003.
[43]冯丹.外存储系统并行性研究. [博士学位论文].武汉:华中科技大学图书馆, 1997.