基于SAN的安全单映像Cluster文件系统关键技术的研究
详细信息    本馆镜像全文|  推荐本文 |  |   获取CNKI官网全文
摘要
随着Internet应用的蓬勃发展和宽带的普及,以及多媒体、视频图像处理技术的大量应用,传统的UNIX文件系统已无法满足大规模或超大规模文件安全存储、高速传输的需要,无法适应大规模实时并发访问的要求,并且在安全性、可扩展性和可用性等方面部存在问题。高性能、高可扩展性、高可用性、安全文件系统日益成为网络时代最重要的关键技术之一。
     针对上述问题,本课题研究并实现了基于SAN(Storage Area Network)的高性能、单映象、安全Cluster文件系统——CFS。
     本文对比SAN存储架构同传统存储架构DAS(Direct Attached Storage)、NAS(Network Aaached Storage)的不同,分析了不同存储架构的优缺点;同时介绍单映像Cluster文件系统的基本概念,及其研究设计所涉及的关键技术。
     本文对系统可用性进行了分析,给出了可用性的度量定义和数学分析;在此基础上,分析、设计了系统NIC(Network Interface Card)级的高可用性,实现了Linux操作系统中提供NIC级高可用性的软件;最后提出了RAIN(Redundant Array of Inexpensive NIC)的概念,详细讨论了其技术实现方案,并给出了Linux下的参考实现。
     本文详细讨论了Cluster系统中成员(membership)的管理问题,在分析集中式管理方法和分布式管理方法特点的基础上,提出了基于SAN的成员管理算法,并对其特点和性能进行了分析。
     本文深入研究了单映像文件系统中锁的结构,提出并设计了分布式环境下动态锁结构,同时提出了可变粒度锁的概念,提出并设计、实现了可变粒度锁的分裂、融合算法。
     本文对分布式环境下锁的管理方法展开研究,分析了集中式锁管理算法,提出并实现了可迁移的共用锁服务器算法,并对其特点进行了讨论。提出了基于SAN的分布式锁管理算法.该算法协议简洁,可扩展性好。
     本文设计了32位安全处理器,针对Cluster文件系统应用特点提出了相应的体系结构。在确定安全处理器体系结构的基础上,设计并实现3DES加密协处理器和高性能的真随机数发生器:同时针对新的指令集结构完成了Linux环境下编译开发环境的设计与实现,将GNU的GCC编译环境成功移植到32位安全处理器上。
     本文最后讨论、总结CFS各个层面的实现方法、特点,并对该文件系统的性能进行了多方面的测试,对测试结果进行分析。
     CFS文件系统已经通过权威测试机构的测试,测试结果表明CFS文件系统性能优异,目前CFS文件系统已经在一些行业和领域取得了实际应用,具有很好的市场价值和前景。
In recent years, Internet has taken a more and more important role in our life, and with the boost of the wide-band, multi-media application and VOD, the traditional UNIX File system can not meet the need of the secure storage and high speed transmission for millions files, and they don't support large-scale, real-time and parallel accesses to the files. At the same time, traditional UNIX File system has a lot of problem in security, extensibility and availability. File System with high performance, high extensibility, high availability and high security has been one of the most important technologies of the Internet.In this project, we take account of the problem above, research and realize a SAN based high performance, high extensibility, high availability and high security Cluster File System-CFS.This paper discusses the difference of the storage architectures-DAS, NASand SAN, and analyzes the advantage of the SAN architecture. This paper also gives the concepts and key technology of the single image Cluster file system.This paper analyzes system's availability, presents the definition of the availability and quantitative analysis of system's availability. According this, we have designed and realized the system's high availability at NIC level, and released the software in Linux. At last we presents the theory of the RAIN (Redundant Array of Inexpensive NIC), and gives the reference implementation in Linux.This paper discusses the membership management in cluster system, based the analysis of central method and distributed method, we presents a new membership management algorithm based SAN, and analyzes the algorithm's feature and performance.This paper presents the dynamic lock structure in distributed system and the concept of the dynamic granularity lock, esigns and realizes the algorithm of the shrink and expands of the granularity.This paper researches the lock management in distributed system, analyzes the central management algorithm, presents and realizes moveable shared lock server. We also presents a new distributed lock management algorithm based SAN, the protocol of the algorithm is simple, and the extensibility is strong.This paper designs a new 32-bit security processor, we presents the architecture according the features of the cluster file system, and presents a new instruction set for the processor. Based on the architecture, we designs a 3DES encrypt co-processor and a TRNG (True Random Number Generator); Based on the instruction set, we import the GCC for our processor.At last, this paper discusses the CFS's realization and the test result.Now, CFS has been tested by the famous test company-TESCOM, the resultshow the CFS is high performance. CFS has been used in some field and company, it has good market value and bright foreground.
引文
[1] Thomas M. Ruwart, "OSD: A Tutorial on Object Storage Devices", In the Tenth NASA Goddard Conference on Mass Storage Systems and Technologies Nineteenth IEEE Symposium on Mass Storage Systems College Park, Maryland, USA. April, 2002
    [2] G. A. Gibson and et al. "File server scaling with network attached secure disks", In Proceedings of the ACM SIGMETRICS, June 1997
    [3] D. Nagle, G. Ganger, J. Butler, G. Goodson, and C. Sabol. "Network Support for Network-Attached Storage", In Proc. of Hot Interconnects, Stanford, CA, August 1999
    [4] E. Riedel and G. Gibson. "Active disks ?Remote execution for network-attached storage", Technical Report CMUCS -97-198,, School of Computer Science, Carnegie Mellon University, PA, 1997
    [5] H. Yokota, "Performance and reliability of secondary storage systems", In Proc. of World Multiconference on Systemics, Cybernetics and Informatics, 668-673, Jul 2000
    [6] Steven Wilson, "Managing a Fibre Channel Storage Area Network", Storage Network Management Working Group for Fibre Channel (SNMWG-FC) November20,1998
    [7] Prasenjit Sarkar, Kaladhar Voruganti, "IP Storage: The Challenge Ahead", In the Tenth NASA Goddard Conference on Mass Storage Systems and Technologies Nineteenth IEEE Symposium on Mass Storage Systems College Park, Maryland, USA, April, 2002
    [8] D. A. Patterson, G. Gibson, and R. H. Katz. "A Case for Redundant Array of Inexpensive Disks (RAID)", In Proceedings of ACM SIGMOD, 109-116, Chicago, IL, June 1988
    [9] Benner, A. "Fibre Channel: Gigabit Communications and I/O for Computer Networks", McGraw Hill, New York, 1996
    [10] Cummings, Roger, "System Architectures Using Fibre Channel", Proc. 12th IEEE Symposium on Mass Storage Systems, pp.251-256, 1993
    [11] Marshall K. McKusick, William N. Joy, Samuel J. Leffler, and Robert S. Fabry. "A fast file system for UNIX". Transactions on Computer Systems, 2(3): 181-197, 1984
    [12] McKusick, M. K.., Karels, M., and Leffler, S. J., "Performance Improvement and Functional Enhancements in 4.3BSD", In Proceedings of the Summer USENIX Conference, Jun. 1985, pp.519-531
    [13] K. A. Smith and M. Seltzer. "A comparison of FFS disk allocation policies", In roceedings of the USENIX 1996 Annual Technical Conference, San Diego, CA, pages 15-25, Jan. 1996
    [14] Andrew S. Tanenbaum, "Operating Systems: Design and Implementation", Prentice-Hall, Inc., Englewood Cliffs, New Jersey, 1987
    [15] Remy Card, Theodore Ts'o, Stephen Tweedie, "Design and Implementation of the Second Extended Filesystem", In Proceedings of the First International Symposium on Linux, Dutch, Dec 1994
    [16] Dave Poirier, "The Second Extended File System: Internal Layout", Boston, MA 02111-1307 USA, 2001
    
    [17] Daniel P. bovet, Marco Cesati, "Understanding the linux kernel", O'Reilly and Associates, Inc, Sabastopol, CA, 2001
    [18] Remy Card. "The Second Extended File System - current state, future development". Second International Linux and Internet Conference, Berlin, May 1995
    [19] Paul Stoffregen, "Understanding FAT32 Filesystems", Located at http://www.pirc.com/tech/8051/ide/fat32.html
    [20] Rajeev Nagar. "Windows NT File System Internals". O'Reilly and Associates, Inc, Sabastopol, CA, 1997
    [21] Werner Vogels. "File System Usage in Windows NT", In Proceedings of the Seventeenth ACM Symposium on Operating Systems Principles, 93-109, Kiawah Island, SC, December 1999
    [22] Danilo Almeida. "Framework for Implementing File Systems in Windows NT", Master's thesis, Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, May 1998
    [23] FreeBSD Documentation Project, "FreeBSD Handbook" 2003, Located at http://www.freebsd.org/doc/en US.ISQ8859-l/books/handbook/index.html
    [24] FreeBSD Documentation Project, "FreeBSD Developer's Handbook" 2003, http://www.freebsd.orp/doc/en US.ISO8859-l/books/developers-handbook/index.html
    [25] Federico Lupi, "The NetBSD Operating System: a Guide", Located at http://www.netbsd.org/guide/en/
    [26] Nick Holland, "OpenBSD Documentation and Frequently Asked Questions", Located at http://www.openbsd.org/faq/index.html
    [27] Kleiman, S. "Vnodes: An Architecture for Multiple File System Types in Sun UNIX", USENIX Summer '86 Con- ference Proceedings, 1986, pp. 238-247
    [28] Michael J. Karels, Marshall Kirk McKusick, "Toward a Compatible Filesystem Interface", In Proceedings of the European Unix User's Group, EUUG, September 1986
    [29] John, A., "Dynamic Vnodes-Design and Implementation", In Proceeding of the Winter 1995 UNSNIX Conference, Jan, 1995, pp.11-23
    [30] Sandberg, R., D. Goldberg, S. Kleinman, D. Walsh and B. Lyon, "Design and Implementation of the Sun Network Filesystem", Usenix Conference Proceedings, June 1985, pp.119-130
    [31] M. Satyanarayanan. "A Survey of Distributed File Systems", Research Report CMU-CS-89116, CS Department, Carnegie-Mellon University (1989)
    [32] R. Sandberg, "The Sun Network Filesystem: Design, Implementation, and Experience", in Proceedings of the Summer 1986 USENIX Technical Conference and Exhibition
    [33] R. Sandberg, "Sun Network Filesystem Protocol Specification", Technical Report, Sun Microsystems, Inc, 1985
    [34] Satyanarayanan, M., Howard, J. H., "The ITC Distributed File System: Principles and Design", Tenth ACM Symposium on OS Principles, Dec, 1985, pp.35-50
    [35] Sidebotham, R. N., "VOLUMES桾he Andrew File System Data Structuring Primitive", In Proceeding of the Autumn 1986 European UNSNIX Conference, Oct 1986, pp.61-70
    
    [36] Gerber, B., "AFS: A Distributed File System that Supports World-wide Networks", Network Computing, May, 1992, pp. 142-148
    [37] Stolarchuk, M. T., "Faster AFS", In Proceeding of the Winter 1993 USENIX Conference, Jan, 1993, pp.67-75
    [38] Levy, E., and Silberschatz, A., "Distributed File System: Concepts and Examples", ACM Computing Surveys, Vol. 22, No.4, Dec, 1990, pp.321-374
    [39] Satyanarayanan, M., Kistler, J. J., Kumar, P., Okasaki, M. E., Siegel, E. H., Steere, D. C., "Coda: a Highly Available File System for a Distributed Workstation Environment", IEEE Transactions on Computers, Vol. 39, No. 4, April 1990
    [40] Kistler, J. J., Satyanarayanan, M., "Disconnected Operation in the Coda File System". ACM Transactions on Computer Systems, Vol. 10, No, 1, February 1992
    [41] M. Satyanarayanan, M. R. Ebling, J. Raiff, and P. J. Braam. "Coda File System User and System Administrators Manual", School of Computer Science, Carnegie Mellon University, August 1997
    [42] Braam, P. J, "The Coda Distributed File System", Linux Journal, June 1998
    [43] P. Braam, M. Callahan, and P. Schwan, "The intermezzo file system", In Proceedings of the 3rd of the Perl Conference, O'Reilly Open Source Convention, Monterey, CA, USA, Aug. 1999
    [44] Braam, P. J., Nelson, P. A, "Removing Bottlenecks in Distributed Filesystems: Coda & InterMezzo as examples", In Proceedings of Linux Expo, May, 1999
    [45] 史小冬,孟丹,祝明发,“COSMOS:一种可扩展单—映象机群文件系统”,南京大学学报,2001年10月
    [46] 贺劲,徐志伟,孟丹,马捷,冯军.基于高速通信协议的COSMOS分布文件系统性能研究,计算机研究与发展,2002年2月,Vol 39,No 2
    [47] Cong Du and Zhiwei Xu, "Performance analysis of a cluster file system", In Proceedings of the 7th International Conference on Parallel and Distributed Processing Techniques and Applications, Las Vegas, 2000, PP1933-1938
    [48] S. R. Soltis, T. M. Ruwart, and M. T. O'Keefe. "The global file system", In proceedings of the 5th NASA Goddard Mass Storage Systems and Technologies Conference, College Park, MD., September 1996
    [49] S. R. Soltis, G. M. Erickson, K. W. Preslan, M. T. O'Keefe, and T. M. Ruwart. "The Global File System: A File System for Shared Disk Storage", Submitted to the IEEE Transactions on Parallel and Distributed Systems, 1997
    [50] K. W. Preslan, A. Barry, J. Brassow et al, "Implementing Journaling in a Linux Shared Disk File System", In Proceedings of the Seventeenth IEEE Symposium on Mass Storage Systems, 351-378, College Park, Maryland, March, 2000
    [51] F. Schmuck and R. Haskin, "GPFS: a shared-disk file system for large computing clusters", In Proceedings of FAST '02, January 2002
    [52] Network Appliance Inc., "NearStore R150 System", Located at http://www.netapp.com/products/nearstore/r150 ds.html?fmt=print
    [53] Microsoft. Common internet file system (CIFS). Located at http://www.ubiqx.org/cifs/
    [54] Christopher R. Hertel, "Implementing CIFS-The Common Internet FileSystem", Prentice-Hall Inc. Englewood, Cliffs, N J, 2002
    
    [55] Marc Farley, "Building Storage Networks", McGraw-Hill Osborne Media, 2nd edition 2001
    [56] Sun, "Sun TrunkingTM 1.2 Software: Just the Facts", Sun Microsystems, Inc. 1999
    [57] Richard Steven, "UNIX Network Programming Volume 1", Prentice Hall, Inc., Englewood Cliffs, New Jersey, 1998
    [58] Alessandro Rubini, "Linux Device Drivers", O'Reilly and Associates, Inc, Sabastopol, CA, 1998
    [59] Douglas E. Comer, "Internetworking With TCP/IP Vol I", Prentice Hall, Inc., Englewood Cliffs, New Jersey, 1995
    [60] Y. Amir, D. Dolev, S. Kramer, and D. Malki. "Membership Algorithms for Multicast Communication Groups", In 13th IEEE ICDCS, pp. 551-560, 1993
    [61] M. O. Killijian, R. Cunningham, R. Meier and V. Cahill, "Towards Group Communication for Mobile Participants", in Principles of Mobile Computing (POMC2001), Newport, Rhode Island, USA, 2001, pp. 75-82
    [62] N. Yialelis and M. Sloman. "A security framework supporting domain based access control in distributed systems", In ISOC Symposium on Network and Distributed Systems Security (SNDSS96), pp.26-39, San Diego, California, Feb. 1996. IEEE Press
    [63] T. Anderson, M. Dahlin, J. Neefe, D. Patterson, D. Roselli, and R. Wang, "Serverless Network File Systems", 15th SOSP, ACM, Dec 1995, pp. 109-126
    [64] L. Moser, P. Melliar-Smith, and V. Agrawala. "Membership algorithms for asynchronous distributed systems", In Proceedings of the 11th International Conference on Distributed Computing Systems, pages 480-489, May 1991
    [65] Michael J. Katchabaw, Stephen L. Howard, Andrew D. Marshall, and Michael A. Bauer. "Evaluating the costs of management: A distributed applications management testbed", In Proceedings of CASCON '96, Toronto, Canada, November 1996
    [66] T. Chandra, V. Hadzilacos, S. Toueg and B. Charron-Bost. "On the Impossibility of Group Membership in Asynchronous Distributed Systems", In Proc. 15th ACM Syposium on Principles of Distributed Computing, Philadelphia, PA, pp. 322--330, May 1996
    [67] S. Mishra, L. L. Peterson and R. D. Schlichting, "A membership protocol based on partial order", in J. F. Meyer and R. D. Schlichting (eds), Dependable Computing for Critical Applications 2, SpringerVerlag, Vienna: 1992, pp. 309-331
    [68] Goldzmith G. and Yemini, Y. "Decentralizing Control and Intelligence in Network Management", In the Proc. of 4th International Symposium on Integrated Network Management, Santa Barbara, CA, 1995
    [69] Garcia-Molina, H. "Elections in a distributed computing system", IEEE Transactions on computers, 51,1, Jan 1982, pp. 48-59
    [70] Chan, M.Y. and F. Y. L. Chin, "Distributed election in complete networks", Distributed computing, 2, 1988, pp. 19-22
    [71] Chow, R., K. Luo, and R. Newman-Wolfe, "An optimal distributed algorithm for failure-driven leader election in bounded-degree networks", In Proceeding of IEEE Workshop on future trends of Distributed Computing System, 1992, pp. 136-153
    [72] King, G.T., T.B. Gendreau, and L. M. Ni, "Reliable election in broadcast networks", Journal of Parallel and Distributed Computing, 1, 1989, pp.521-546
    
    [73] Singh, S. and J. Kurose, "Electing 'good' leaders", Journal of Parallel and Distributed Computing, 21,2, 1994, pp. 184-201
    [74] Chang, C. K., "Bidding against competitors", IEEE Transactions on Software Engineering, 16, 1, 1990, pp. 100-104
    [75] Wu, J. and E.B. Fernandez, "An extended bidding scheme for the distributed consensus problem", Technical Report, TR-CSE-92-38, Florida Atlantic University, 1992
    [76] WTI Part No.: 12927, "Network Power Switch User's guide", 5 Sterling, Irvine, California 92618-2517 USA
    [77] Jeffrey Katcher, "PostMark: A New File System Benchmark", TR3022, Network Appliance Inc
    [78] B. Schneier, "Applied Cryptography", John Wiley & Sons, Inc., 1996
    [79] Kenneth Castelino, "3DES and Encryption", Located at http://kingkong.me.berkeley.edu/~kenneth/courses/sims250/des.html
    [80] Kai Hwang, Faye A. Briggs. "Computer Architecture and Parallel Processing". McGrawHill. ISBN 0-07-031556-6. 1984
    [81] Kai Hwang. "Advanced Computer Architecture: Parallelism, Scalability, Programmability". McGraw Hill. ISBN 0-07-031622-8. 1993
    [82] Charles Price, "MIPS IV Instruction Set Revision 3.2", MIPS Technologies, Inc. 1995
    [83] R. M.Cmelik, S. I. Kong, D. R. Ditzel, and E. J. Kelly, "An Analysis of SPARC and MIPS Instruction Set Utilization on the SPEC benchmarks", Proc. ASPLOS-IV, April, 1991, Santa Clara, CA, pp. 290-302
    [84] ARM Inc, ARM Architecture Reference Manual, ARM DDI 0100D, White Paper, 2002
    [85] . S.B. Furber. "ARM System-on-Chip Architecture". Addison Wesley Longman, 2000. ISBN: 0-201-67519-6
    [86] Dave Jagger, David Seal. "ARM Architecture Reference Manual (2nd Edition)". Addison-Wesley Pub Co. ISBN: 0201737191
    [87] Hewlett Packard. PA-RISC 1.1 "Architecture and Instruction Set Reference Manual", Hewlett Packard, February 1994. Third edition
    [88] D. A. Patterson. "Reduced instruction set computers", Communications of the ACM, 28(1):8-21, January 1985
    [89] GNU, GCC home page, Located at http://gcc.gnu.org/
    [90] Microsoft, "Microsoft Portable Executable and Common Object File Format Specification", Microsoft Corporation, Revision 6.0 - February 1999
    [91] Margo Seltzer and Keith Smith. "File system logging versus clustering: A performance comparison", In USENIX Annual Technical Conference, June 1995
    [92] Vahalia, U., Gray, C, Ting, D. "Metadata Logging in an NFS Server", In Proceedings of the USENIX Technical Conference, pp. 265-276. New Orleans, LA, Jan. 1995
    [93] Stephen Tweedie Set, "Journalling the ext2fs Filesystem", In LinuxExpo, 1998
    [94] G. Ganger and Y. Part. "Metadata Update Performance in File Systems", In Proc. of USENIX Symposium on Operating System Design and Implementation, pages 49-60, November 1994
    [95] G. Ganger, M. McKusick, C. Soules, and Y. Part, "Soft updates: a solution to the metadata update problem in file systems", ACM Transactions on Computer Systems, Vol. 18, No. 2, 2000, pp. 127-153
    
    [96] M. K. McKusick and G. R. Ganger. "Soft updates: A technique for eliminating most synchronous writes in the fast Filesystem", In Proceedings of the FREENIX Track, USENIX Annual Technical Conference, pages 1-17, Monterey, California, June 1999
    [97] M. Seltzer, G. Ganger, M. McKusick, K. Smith, C. Soules, C. Stein, "Journaling versus Soft Updates: Asynchronous Meta-data Protection in File Systems", USENIX Conf. Proc, pp. 71-84, San Diego, CA, June 2000
    [98] M. Aron, P. Druschel, and W. Zwaenepoel. "Efficient Support for P-HTTP in Cluster-based Web Servers", In Proceedings of the USENIX 1999
    [99] Tim Bray, Bonnie: file system benchmark, Located at http://www.textuality.com/bonnie/

© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号

地址:北京市海淀区学院路29号 邮编:100083

电话:办公室:(+86 10)66554848;文献借阅、咨询服务、科技查新:66554700