广域网络虚拟文件系统关键技术研究
详细信息    本馆镜像全文|  推荐本文 |  |   获取CNKI官网全文
摘要
以现有的网络及分布式技术为基础,研究广域网尺度的文件资源共享和重用,针对性地提出了虚拟网络文件系统(VIRTUAL NETWORK FILE SYSTEM---VNFS)模型。VNFS统一化管理广域网络上的各类文件资源,为用户提供统一的共享文件系统。
     论文围绕实现VNFS的关键技术问题,包括系统逻辑结构组织、文件全局名字空间服务、副本管理技术、数据一致性技术等展开研究讨论,提出了若干新算法和方法,并进行了相关的原型系统设计和算法性能试验。
     第一章论述了VNFS系统研究的必要性和应用环境。第二章对目前已有研究综述和分析,讨论了目前网络共享文件系统的现状及存在的问题,明确了研究的主要内容。第三章给出本文研究的内容,包括设计一个灵活的、能够逼近于物理网络拓扑的逻辑组织结构;实现VNFS的全局名字空间GNS(Global Name Space);高效的副本管理机制及数据一致性维护机制。
     VNFS结构原型及模块功能描述在第四章中展开,整个VNFS系统按照完全对等的思想设计,任何一个加入VNFS系统的服务节点都安装VNFS功能软件并可以充当其中的任何角色。第五章详细描述了VNFS的逻辑组织构建方法---一个自组织叠加网模型的设计及相应的算法实现,并从理论上证明了其高效性。第六章以自组织算法为基础构建了VNFS的全局名字空间,提出了分别从物理特征、逻辑特征、用户使用特征三个层次对文件命名的方法,并建立三个层次文件名字间的映射关系,提高了名字解析效率。
     第七章给出VNFS的副本管理方法,其针对广域网特征进行了新的设计。提出了适应于局域网和广域网环境的两个副本管理算法,并证明了算法在提高文件访问服务质量方面的良好性能。第八章给出应用于VNFS系统中的一种动态数据同步方法以及一个分布式互斥锁算法。该动态数据同步方法综合了扩散机制和失效机制的特点,提高了同步效率;该分布式互斥算法基于请求的读写特征设计,降低了系统分布式互斥的通信开销。
Based on the network and distributed technology, a model named virtual network file system(VNFS),which unites all the physical file systems into one virtual network file system to the users, is presented to research the ways of sharing and reusing the files on the wide area.
    The key problems in VNFS, such as system architecture, global name space, replication management and data consistency etc. will be discussed in this thesis. Some new algorithms and ideas, whose performances are analyzed and original modal is designed, will also be presented.
    In chapter one, I explain why we need to research VNFS and what the application environment is. In chapter two, I further analyze and summarize the actualities and disadvantage of the network file system and determine the key problems of research. In chapter three, the main research contents are described, including: designing a logical architecture which is flexible and approachable to physical network, completing GNS(Global Name Space) of VNFS, implementing a high-efficient replication management algorithms and data coherence algorithms.
    In chapter four, the architecture model of VNFS and the functions of every module are described. The VNFS is designed as the peer-to-peer model and every site in VNFS can play any role. In chapter five, a hierachical self-organizing overlay network model based on clustering, which is the base of VNFS logical organization, is proposed and the efficiency of this model is proved theoretically. In chapter six, the GNS of VNFS is designed based on the self-organizing overlay network model. A three layers name space, which are physical layer, logical layer and application layer, is proposed, and the relations among the three layers are presented. Using this name space we can accelerate the process of looking up a file.
    In chapter seven, the methods of VNFS replication management are presented, which are based on the replication management methods in distributed file systems and expand some new characteristics to fit the WAN environments. Two replication
引文
[1] I. Foster and C. Kesselman. Globus: A metacomputing infrastructure toolkit. The International Journal of Supercomputer Applications and High Performance Computing, 11 (2): 115-128, 1997
    [2] Chaitanya Baru, Reagan Moore, Arcot Rajasekar, and Michael Wan. The sdsc storage resource broker. In Proceedings of IBM Centers for Advanced Studies Conference. IBM, 1998.
    [3] Osamu Tatebe, Youhei Morita, Satoshi Matsuoka, Noriyuki Soda, and Satoshi Sekiguchi. Grid datafarm architecture for petascale data intensive computing. In Henri E. Bal, Klaus-Peter L¨ohr, and Alexander Reinefeld, editors, Proceedings of the Second IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid2002), pages 102-110, Berlin, Germany, 2002. IEEE, IEEE Computer Society.
    [4] M. Satyanarayanan, J. H. Howard, D. N. Nichols, R. N. Sidebotham, A. Z. Spector and M. J. West, The ITC Distributed File System: Principles and Design, Proceedings of the loth Symposium on Operating System Principles (SOSP), Orcas Island, Washington, U.S., ACM Press, December 1985.
    [5] Howard, J.H. An Overview of the Andrew File System, Proceedings of the USENIX Winter Technical Conference Feb. 1988, Dallas, TX
    [6] S. Shepler, B. Callaghan, D. Robinson, R. Thurlow, C. Beame, M. Eisler, and D. Noveck, "Network File System (NFS) Version 4 Protocol", RFC3530, Internet Engineering Task Force (April 2003), http://www.ietf.org/rfc/rfc3530.txt
    [7] Satyanarayanan, M. Integrating Security in a Large Distributed System, ACM Transactions on Computer Systems, Aug. 1989, Vol. 7, No. 3, pp. 247-280
    [8] Howard, Kazar, M.L. J.H. Scale and Performance in a Distributed File System, ACM Transactions on Computer Systems Feb. 1988, Vol. 6, No. 1, pp. 51-81.
    [9] Satyanarayanan, M. Scalable, Secure, and Highly Available Distributed File Access, IEEE Computer, May 1990, Vol. 23, No. 5
    [10] Braam, P. J. The Coda Distributed File System, Linux Journal, June 1998.
    [11] Satyanarayanan, M. Coda: A Highly Available File System for a Distributed Workstation Environment, Proceedings of the Second IEEE Workshop on Workstation Operating Systems Sep. 1989, Pacific Grove, CA
    [12] Peter Braam, Philip Nelson, Removing Bottlenecks in Distributed Filesystems: Coda InterMezzo as examples, www.inter-mezzo.org..
    [13] Michael N. Nelson, Brent B. Welch, and John K. Ousterhout, Caching in the Sprite Network File System. ACM Transactions on Computer Systems, 6(1), February 1988. pp134-154.
    [14] Brent B. Welch: Measured Performance of Caching in the Sprite Network File System. Computing Systems 4(3): pages 315-342,1991.
    [15] Fred Douglis, John K. Ousterhout, M. Frans Kaashoek, Andrew S. Tanenbaum, A Comparison of Two Distributed Systems: Amoeba and Sprite, ACM Transactions on Computer Systems, 4(4), Fall 1991..
    [16] E. K. Lee and C. Thekkath. Petal: Distributed virtual disks. In Proceedings of the 7th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS-Ⅶ), pages 84-92, October 1996.
    [17] Chandramohan A. Thekkath, TimothyMann, Edward K. Lee, Frangipani: A Scalable Distributed File System, Symposium on Operating Systems Principles (SOSP), 1997.
    [18] 史小冬,孟丹,祝明发,COSMOS:一种可扩展单一映象机群文件系统,南京大学学报(自然科学),2001.10
    [19] 吴思宁,贺劲,熊劲,孟丹,DCFS机群文件系统服务器组的设计与实现,2002全国开放式与并行计算学术会(DPCS2002),2002.
    [20] Randolph Y. Wang and Thomas E. Anderson. xFS: A Wide Area Mass Storage File System. In Proceedings of the Fourth Workshop on Workstation Operation Systems, pages 71—78, October 1993.
    [21] Darrell C. Anderson, Jeffrey S. Chase, Amin M. Vahdat, Interposed Request Routing for Scalable Network Storage, Proceedings of the Fourth Symposium on Operating System Design and Implementation, October 2000.
    [22] Steve Soltis, Grant Erickson, Ken Preslan, Matthew O'Keefe, and Tom Ruwart, The Design and Performance of a Shared Disk File System for IRIX, Fifteenth IEEE Symposium on Mass Storage Systems, 1998.
    [23] Kenneth W. Preslan, Andrew P. Barry, etc. A 64-bit, Shared Disk File System for Linux, Storage Conference, 1999.
    [24] Frank Schmuck and Roger Haskin, GPFS: A Shared-Disk File System for Large Computing Clusters, Proceedings of the Conference on File and Storage Technologies (FAST'02). 28-30 January 2002, Monterey, CA, pp. 231—244. (USENIX, Berkeley, CA.).
    [25] J. Menon, D. A. Pease, R. Rees, L. Duyanovich, B. Hillsberg, IBM Storage Tank. A heterogeneous scalable SAN file system, IBM SYSTEMS JOURNAL, VOL 42, NO 2, 2003.
    [26] PANASAS WHITE PAPER, Object Storage Architecture, http://www.panasas.com, 10/19/2003.
    [27] www.lustre.org
    [28] A. Grimshaw, "Enterprise Data Grids," Tutorial presentation at Global Grid Forum 7 (GGF7), Tokyo, Japan (March 2003), http://www.gfidforum.org/Meetings/ggf7/tutl.htm
    [29] Roger Haskin and Frank Schmuck, The Tiger Shark File System, Proceedings of IEEE 1996 Spring COMPCON, Santa Clara, CA, Feb. 1996.
    [30] Roger Haskin, The Shark Continuous Media File Server, Proceedings of IEEE 1993 Spring COMPCON, San Fransisco, CA, Feb. 1993, pp. 12-17.
    [31] Roger Haskin and Frank Stein, A System for the Delivery of Interactive Television Programming, Proceedings of IEEE 1995 Spring COMPCON, San Fransisco, CA, Mar. 1995, pp. 209-214.
    [32] 何炎祥.分布式操作系统设计.北京:海洋出版社 1993
    [33] Lightweight directory access protocol(v3)(RFC2251)
    [34] Common Internet File System Technical Reference, Storage Networking Industry Organization, http://www.snia.org/tech_activities/CIFS/
    [35] R. Bums. Data management in a Distributed File System for Storage Area Networks. PhD Thesis. Department of Computer Science, University of California, Santa Cruz, March 2000.
    [36] 石柯等,数据网格中一致基于副本和缓存的元数据管理系统.计算机研究与发展2004,第41卷,第12期,pp2206-2210
    [37] M. Waldovgel and R. Rinaldi. Efficient Topology-Aware Overlay Network[C]. ACM HotNets 2002, SIGCOMM/CCR 2003.
    [38] Ratnasamy S. Handley M. KarpR. Shenker S. Topologically-aware overlay construction and server selection[C]. INFOCOM 2002. Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings. IEEE,Volume 3,23-27 June 2002 Pages:1190-1199 vol.3.
    [39] Sylvia Ratnasamy, P.Francis,M. Handley, and R. Karp. A Scalable Content-Addressable Network[C]. In Proceedings of SIGGCOMM 2001,Pages 161-172,San Diego,CA,USA,August 2001.
    [40] Zhichen Xu, Chunqiang Tang, Zheng Zhang. Building topology-aware overlays using global soft-state[C]. Distributed Computing Systems,2003. Proceedings. 23rd International Conference on,2003 Pages:500-508.
    [41] Xin Yan Zhang; Qian Zhang; Zhensheng Zhang; Gang Song; Wenwu Zhu; A construction of locality-aware overlay network: mOverlay and its performance Selected Areas in Communications[J], IEEE Journal on, Vol: 22,Issue: 1,Jan. 2004 Pages: 18-28.
    [42] BRITE.[Online].Available: http://www.cs.bu.edu/brite/.
    [43] A-L. Barabasi, R.Albert. Emergence of scaling in random networks[J], Science, 286,1999: 509-512.
    [44] P Erdos, A Renyi: On random graphs[M]. I. Publ. Math. (Debrecen) 1959,6:290-297.
    [45] E Christensen, F Curbera, G Meredith et al. Web services description language(WSDL) 1.1. W3C(World Wide Web Consortium) [Online].Cambridge,MA,2001. http://www.w3.org/TR/wsdl
    [46] Random graphs[m], by Bollobas,Bela. Academic Press,inc. [Harcourt Brace Jovanovich,Publishers], London-New York (1985) 1-447
    [47] D. J. Watts,S. H. Strogatz. Collective dynamics of"small world" networks[J]. Nature, 393, 1998:440-442
    [48] P. Mockapetris, Domain Names—Concepts and Facilities, Nov. 1987, RFC 1034.
    [49] P. Mockapetris, Domain names—Implementation and Specification, Nov.1987, RFC 1035.
    [50] P. Mockapetris and K. Dunlap, "Development of the Domain Name System,"in Proceedings of the ACMSIGCOMM, Stanford, CA, 1988, pp. 123-133.
    [51] A L Chervenak, I foster, C kessel man et al. Data management and transfer in high performance computational grid environments. Parallel Computing Journal, 2002, 28(5):749-771
    [52] H Sotckinger, A Samar, B Allcock et al. File and object replication in data grids. Journal of Cluster Computing, 2002, 5(3): 305-314.
    [53] M Ripeanu, I Foster. A decentralized, adaptive, replica location service. In: Proc of HPDC-11. Edinburgh, Scotland: IEEE Computer Society Press, 2002.
    [54] A Chervenak, E Deelman, I Foster et al. Giggle: A framework for constructing scalable replica location services. In: Proc of Supercomputing 2002(SC2002). Baltimore, USA:IEEE Computer Soceity Press, 2002.
    [55] D. W. Brubeck and L. A. Rowe, "Hierarchical storage management in a distributed Video-On-Demandsystem." IEEE Multimedia, Volume: 3 Issue: 3, Fall 1996 pp.37-47
    [56] C. C. Bisdikian and B. V. Patel "Cost-based program allocation for distributed multimedia-on-demand systems,"IEEE Multimedia Fall, pp.62-72,1996.
    [57] Kyung-Ah AHN and Hoon CHOI "Architecture of a BOD System with Proxy Servers,"IEICE TRANS. COMMUN.,VOL.E83-B,NO.4 APRIL 2000
    [58] 李勇、吴非、陈福接“大规模层次化视频点播存储系统的设计与管理”软件学报第10卷第4期 1999年4月
    [59] Zhi-Li Zhang, Yue wei Wang, Video Staging "A proxy-server-Based Approach to End-to-End Video Delivery over Wide-Area Networks. Networking." IEEE/ACM Transactions on, Volume: 8 Issue: 4, Aug. 2000.pp.429-442
    [60] Hongtao Yu; Chor Ping Low; Atif, Y. "Design issues on video-on-demand resource management."Networks, 2000. (ICON 2000). Proceedings. IEEE International Conference on, 2000. pp. 199-203
    [61] Kit-Sang Tang; King-Tim Ko; Chan, S.; Wong, E.W.M. "Optimal file placement in BOD system using genetic algorithm" Industrial Electronics, IEEE Transactions on, Volume: 48 Issue: 5, Oct. 2001 Page(s): 891-897
    [62] Russel Sandberg, "The Sun Network File System: Design, Implementation and Experience", Proceedings of the 1987 Summer Usenix Conference, University of California Press, Summer 1987, pp. 300-314.
    [63] 沈海华,沈美明等,WWW集群服务器的数据方便分布方式研究.软件学报 2001,第12卷,第3期,pp367-371.
    [64] T. E. Anderson, M. D. Dahlin, J. M. Neefe, D. A. Patterson, "Serverless Network File Systems", Proceedings of the 15th ACM Symposium on Operating Systems Principles, 1995.
    [65] J. H. Morris, M. Satyanarayana, M. H. Conner, J. H. Howard, "Andrew: A distributed personal computing environment", Commun. ACM29, 3, March 1986, pp. 184-201.
    [66] James J. Kistler and M. Satyanarayanan. "Disconnected Operation in the Coda File System". Proceeding of the 13th Symposium on Operating Systems Principles, Pacific Grove, CA, October 1991, pp. 213-225.
    [67] V. Cate. "ALEX—A global file system". Proceedings of the 1992 USENIX File System WORKSHOP(Ann Arbor, MI, May). USENIX Assoc., Berkeley, CA. 1992.
    [68] Amin Vahdat, Paul Eastham, and Thomas Anderson. "WebFS: A Global Cache Coherent Filesystem". Technical Draft, December 1996.
    [69] Jun Wang, Dong Li, "A light-weight, temporary file system for large-scale web servers" Modeling, Analysis and Simulation of Computer Telecommunications Systems, 2003. MASCOTS 2003. 11th IEEE/ACM International Symposium on, 12-15 Oct. 2003 Pages:96-103
    [70] M. Rosenblum and J. K. Ousterhout, "The design and implementation of a log-structured file system," ACM Transactions on Computer Systems, vol. 10, pp. 26-52, Feb. 1992.
    [71] Changpeng Fan; "Realizing a soft real-time framework for supporting distributed multimedia applications" Distributed Computing Systems, 1995., Proceedings of the Fifth IEEE Computer Society Workshop on Future Trends of, 28-30 Aug. 1995 Pages:128-134
    [72] C. Lee, R.Rajkumar, and C. Mercer. "Experiences with Processor Reservation and Dynamic QOS in Real-Time Mach". Proceedings of Multimedia Japan, March 1996.
    [73] P. Goyal, X. Guo, and H. Vin. "A Hierarchical CPU Scheduler for Multimedia Operating Systems". Proceedings of the Second Symposium on Operating Systems Design and Implementation, Seattle, Washington, October 1996
    [74] D. Yau and S. Lain. "Adaptive Rate-Controlled Scheduling for Multimedia Applications". Proceedings of the 1996 ACM Multimedia Conference, Boston, Massachusetts, November 1996.
    [75] Molenkamp, G., Lutfiyya, H., Katchabaw, M., Bauer, M., "Diagnosing quality of service faults in distributed applications" Performance, Computing, and Communications Conference, 2002.21st IEEE International, 3-5 April 2002 Pages:375-382
    [76] Fang Yu, Qian Zhang, Wenwu Zhu, Ya-Qin Zhang, "QoS-adaptive proxy caching for multimedia streaming over the Internet" Circuits and Systems for Video Technology, IEEE Transactions on, Volume: 13, Issue:3, March 2003 Pages:257-269
    [77] J. Chuang, M. Sirbu, Pricing multicast communication: a cost based approach, in: Proceedings of INET' 98, Geneva, Switzerland,1998.
    [78] R. C. Chalmers, K. C. Almeroth, Modelingthe branching characteristics and ef.ciency gains in global multicast trees, in:Proceedings of INFOCOM' 2001, Anchorage, AK, 2001.
    [79] Lodha, S.; Kshernkalyani, A. "A fair distributed mutual exclusion algorithm" Parallel and Distributed Systems, IEEE Transactions on, Volume: 11 Issue: 6, June 2000 Page(s): 537-549
    [80] Y.-I. Chang,"A Simulation Study on Distributed Mutual Exclusion," J. Parallel and Distributed Computing, vol. 33,pp. 107-121 1996
    [81] M. Singhal, "A taxonomy of Distributed Mutual Exclusion," J. Parallel and Distributed Computing, vol. 18,no.l, pp.94-101, May 1993
    [82] J. Helary, A. Mostefaoui, and M. Raynal, "A General Scheme for Token-and Tree-Based Distributed Mutual Exclusion Algorithms," IEEE Trans. Parallel and Distributed Systems, vol.5, no. 11, pp. 1185-1196, Nov. 1994.
    [83] Y.-C Kuo and S.-T. Huang, "A Geometric Approach for Constructing Coteries and K-Coteries," IEEE Trans. Parallel and Distributed Systems, vol.8,no.4, pp.402-411, Apr.1997.
    [84] M. Naimi and M. Trehel, "An Improvement of the Log(n) Distributed Algorithm for Mutual Exclusion," Proc. Seventh Int'l Conf. Distributed Computing System, pp.371-375,1987
    [85] L. Lamport, "Time, Clocks and Ordering of Events in Distributed Systems," Comm. ACM, vol. 21, no. 7, pp. 558-565,July 1978.
    [86] G. Ricart and A. K. Agrawala, "An Optimal Algorithm for Mutual Exclusion in Computer Networks," Comm. ACM, vol. 24, no. 1, pp. 9-17, Jan. 1981.
    [87] M.Singhal, " A Dynamic Information Structure Mutual Exclusion Algorithm for Distributed Systems," IEEE Trans. Parallel and Distributed Systems, Jan. 1992.
    [88] O. Carvalho and G. Roucairol, "On Mutual Exclusion in Computer Networks, TechnicalCorrespondence," Comm. ACM, vol. 26, no.2, pp.146-147, Feb. 1983
    [89] M. Maekawa, "A N~(1/2) Algorithm for Mutual Exclusion in Decentralized Systems," ACM Trans. Computer Systems, May 1985.
    [90] W.-S. Luk and T.-T. Wong, "Two New Quorum Based Algorithms for Distributed Mutual Exclusion," Proc. 17th Int'l Conf. Distributed Computing Systems, May 1997.
    [91] D.Peleg and A. Wool, "Crumbling Walls: A Class of Practical and Efficient Quorum System," Distributed Computing, vol.10, no.2, pp. 120-129, 1997.
    [92] Guohong Cao; Singhal, M. "A delay-optimal quorum-based mutual exclusion algorithm for distributed systems" Parallel and Distributed Systems, IEEE Transactions on, Volume: 12 Issue: 12, Dec. 2001 pp. 1256-1268

© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号

地址:北京市海淀区学院路29号 邮编:100083

电话:办公室:(+86 10)66554848;文献借阅、咨询服务、科技查新:66554700