网格数据资源的层次化发现方法研究
详细信息    本馆镜像全文|  推荐本文 |  |   获取CNKI官网全文
摘要
网格计算是当前分布计算研究领域的热点,其核心是网格资源管理,而网格资源发现则是网格资源管理中一个基本组成部分,它为网格资源调度寻找满足应用需求的各种资源。随着网格的快速发展,集中式的资源发现方式将会出现性能瓶颈。因此,网格的资源发现方式应该是非集中的方式,才能够适应大规模、动态的网格环境。
     数据网格是当前应用领域中的一类重要的网格,实现了广域范围内数据资源的共享以及数据处理能力的共享。本文基于数据网格的特点,提出了一种分布式层次化数据资源发现方法。具体地,本文做了以下几个方面的工作:
     1.层次化网格数据资源发现方法框架
     建立了网格数据资源发现的理论分析模型和分析指标,并与现有方法进行了比较,提出了一种层次化的网格数据资源发现方法框架。该方法框架由两个层次构成,即上层的数据资源类型索引网络,和下层不同类型数据的底层资源网络。基于这种框架,可以有效地限制资源发现请求以及资源信息更新消息的扩散范围,以达到提高资源发现效率的目的。
     2.基于元数据的资源类型索引网络
     提出了一种基于网格元数据的数据资源类型组织方法,并讨论了分布式元数据结点发现、覆盖拓扑构造与维护,以及元数据注册和元数据请求转发方法。
     3.基于局部信息的资源定位方法
     提出了一种基于局部信息的资源定位方法,用于在底层资源网络中搜索满足给定需求的全局较优的资源。我们还建立数学模型分析了该方法的效果,结果表明在一定条件下,该方法能够以较小的消息与时间开销在相当规模的数据网格中找到全局较优的资源。
Nowadays, Grid Computing is a hot research topic in distributed computing area. Resource management is acknowledged as one of the main points of Grid Computing, and resource discovery is a basic issue in Grid resource management, which concerns discovering resources in Grid to meet the requirement of applications. With the rapid development of Grid, centralized Grid resource discovery schemes have potential scalability and performance problems. Therefore, in order to survive the dynamic and larger-scale Grid environment, resource discovery should be decentralized and should not rely on centralized control.
     Data Grid is one of most important Grids, and it provides sharing of both data. resources and the capability of data processing globally. The thesis thus focuses on the Data Grid, and presents a distributed and hierarchical data resource discovery approach. Its distinguished feature is the high efficiency of the resource discovery process while keeping completely distributed.
     In details, the thesis makes following contributions:
     1. The framework of the distributed and hierarchical data resource discovery approach.
     The thesis first constructs the theoretical model for data resource discovery. Then a hierarchical framework for data resource discovery, as well as the comparison with current methods, is presented. The presented framework may effectively control the propagation of messages.
     2. The metadata-based resource type index network.
     Different types of data resources are organized into different resource sub-networks based on their metadata. We also discuss the topics on distributed metadata nodes discovery, topology construction and maintenance, metadata registration, and metadata request forwarding.
     3. The resource locating method inside the underlaid data resource network based on local information.
     Such resource locating method is used to find out the optimal data resource inside the underlaid data resource network. An analysis model is built to study the effectiveness of the presented method, and results are also validated by simulation. It is shown that, this method can find resources with relative high qualities among all qualified resources in a big resource network with small number of hops.
引文
[1] Foster I, Kesselman C, The Grid 2: Blueprint for a New Computing Infrastructure, 2nd Edition, Morgan Kaufmann, 2004.
    
    [2] Foster I, The Grid: Computing Without Bounds, Scientific American, April, 2003.
    [3] Iamnitchi A, Foster I, On Fully Decentralized Resource Discovery in Grid Environments. In: Proceedings of 2nd IEEE/ACM International Workshop on Grid Computing 2001, Denver, November 2001.
    [4] Iamnitchi A, Resource Discovery in Large Resource-Sharing Environments, PHD thesis, University of Chicago, 2003.
    [5] Foster I, Kesselman C, Tuecke S, The Anatomy of the Grid: Enabling Scalable Virtual Organizations, International Journal of Supercomputer Applications, 2001, 15(3).
    
    [6] http://www.nees.org
    [7] http://www. ipg.nasa.gov
    [8] http://eu-datagrid.web.cern.ch/eu-datagrid/
    [9] http://www.disa.mil/ns/gig.html
    [10] http://www.globus.org
    
    [11] Foster I, What is the Grid, Grid Today, 2002,1(6).
    [12] Bote-Lorenzo M, Dimitriadis Y, Gomez-Sanchez E, Grid Characteristics and Uses: a Grid Definition, In: Proceedings of 1st European Across Grids Conference, Spain February, 2003.
    [13] Foster I, Kesselman C, Nick J, Grid Services for Distributed System Integration, IEEE Computer, 2002, 35(6).
    [14] Foster I, Kesselman C, Nick J, The Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Integration, Open Grid Service Infrastructure WG, Global Grid Forum, June 2002.
    [15] http://www.w3 .org/2002/ws/#drafts
    [16] Iamnitchi A. On Death, Taxes, and the Convergence of Peer-to-Peer and Grid Computing. In: Proceedings of the 2nd International Workshop on Peer-to-Peer Systems (IPTPS'03). Heidelberg: Springer-Verlag, 2003.
    [17] Fox G, Pallickara S, Support for Peer-to-Peer Interactions in Web Brokering Systems, ACM Ubiquity, 2002,15(3).
    [18] Fox G, Pallickara S, Rao X, Enabling Peer to Peer Grids, Concurrency and Computation: Practice & Experience. ACM JavaGrande ISCOPE 2002 Special Issue.
    [19] Fox G, Lim S, Pallickara S, Pierce M, Message-Based Cellular Peer-to-Peer Grids: Foundations for Secure Federation and Autonomic Services, to be published in Future Generation Computer Systems, http://grids.ucs.indiana.edu/ptliupages/publications/cellularGrid_final.pdf
    [20] Wang M, Fox G, Pallickara S, A Demonstration of Collaborative Web Services and Peer-to-Peer Grids, In: Proceedings of IEEE ITCC2004 International Conference on Information Technology, Las Vegas, 2004.
    [21]Talia D, Trunfio, Toward a Synergy Between P2P and Grids, IEEE Internet Computing, 2003, July/August Issue.
    [22]Oram A, Peer-to-Peer: Harnessing the Power of Disruptive Technologies, O'Reilly and Associates, 2001.
    [23] http://www.napster.com
    [24] http://www.limewire.com
    [25] http://www.kazza.com
    [26] Anderson D, SETI@Home: An Experiment in Public Resource Computing System, Communications of the ACM, 2002,45(11):56-61.
    [27]Jarek Nabrzyski, Grid Resource Management - State of the Art and Future Trends. Kluwer Academic Publishers, Boston, 2003
    [28]Czajkowskiy K, Fitzgeraldz S, Foster I, Grid Information Services for Distributed Resource Sharing. In: Proceedings of IEEE HPDC-10, 2001.
    [29] Zhang X, Freschl J, Schopf J, A Performance Study of Monitoring and Information Services for Distributed Systems, In: Proceedings of IEEE HPDC-12, 2003.
    [30]Keung H, Dyson J, Jarvis S, Predicting the Performance of Globus Monitoring and Discovery Service In: Proceedings of 4th IEEE/ACM International Workshop on Grid Computing, 2003.
    [31]Plaxton C, Rajaraman R, Richa A, Accessing Nearby Copies of Replicated Objects in a Distributed Environment, In: Proceedings of ACM SPAA, 1997, 311-320.
    [32]Bakker A, Amade E, Ballintijn G, et al, The Globe Distributed Network. In: Proceedings of USENIX Annual Conference, 2000, 141-152.
    [33]Steen M, Hauck F, Ballintijn G, et al., Algorithm Design of the Globe Wide-area Location Service, The Computer Journal, 1998, 41(5): 297-310.
    [34] www.ietf.org/html.charters/svrloc-charter.html
    [35]Tuecke S, Czajkowski K, Foster I, et al., Open Grid Services Infrastructure (OGSI) Version 1.0, Global Grid Forum Draft Recommendation, 2003, http://www. globus. org/research/papers/Final_OGS I_Specification_V 1.0 .pdf
    [36]Thain D, Tannenbaum T, Livny M, Condor and the Grid, In: Berman F, Hey A, Fox G editors, Grid Computing: Making The Global Infrastructure a Reality, John Wiley, 2003.
    [37]Basney J, Livny M, Mazzanti P, Harnessing the Capacity of Computational Grids for High Energy Physics, In: Proceedings of the International Conference on Computing in High Energy and Nuclear Physics (CHEP 2000), Italy, 2000.
    [38] Raman R, Livny M, Solomon M, Resource Management through Multilateral Matchmaking, In: Proceedings of IEEE HPDC-9, Pennsylvania, 2000.
    [39] Raman R, Livny M, Solomon M, Policy Driven Heterogeneous Resource Co-Allocation with Gangmatching, In: Proceedings of IEEE HPDC-12, Seattle , 2003.
    [40]Basney J, Livny M, Managing Network Resources in Condor, In: Proceedings of IEEE HPDC-9, Pennsylvania, 2000.
    [41]Grimshaw A, Ferrari A, Knabe F, Legion: An Operating System for Wide-Area Computing, IEEE Computer, 1999,32(5):29-37
    [42]Grimshaw A, Wulf W, The Legion Vision of a Worldwide Virtual Computer, Communications of the ACM, 1997,40(1)
    [43]Chapin S, Katramatos D, Karpovich J, The Legion Resource Management System, In: Proceedings of the 5th Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP '99), Puerto Rico, 1999.
    [44]UDDI specification, http://www.uddi.org
    [45] The Evolution of UDDI, White Paper of UDDI Organization, http://www.uddi.org/pubs/the_evolution_of_uddi_20020719.pdf
    [46] Lv Q, Cao P, Cohen E. Search and Replication in Unstructured Peer-to-Peer Networks. In: Proceedings of ACM SIGMETRICS 2002.
    [47]Saroiu S, Gummadi P, Gribble S. A Measurement Study of Peer-to-Peer File sharing Systems. In: proceedings of SPIE Multimedia Computing and Networking (MMCN2002), San Jose, 2002.
    [48] Sylvia R, Scott S, Ion S. Routing Algorithms for DHTs: Some Open Questions. In: Proceedings of the 1st International Workshop on Peer-to-Peer Systems (IPTPS '02), Heidelberg: Springer-Verlag, 2002
    [49]Stoica I, Morris R, Karger D, et al., Chord: A Scalable Peer-To-Peer Lookup Service for Internet Applications In: Proceedings of ACM SIGCOMM 2001
    
    [50]Stoica I, Morris R, David Karger D, et al., Chord: A Scalable Peer-to-peer Lookup Protocol for Internet Applications. To Appear in IEEE/ACM Transactions on Networking, http://www.pdos.lcs.mit.edu/chord/papers/paper-ton.pdf
    [51]Ratnasamy S, Francis P, Handley M, A Scalable Content-Addressable Network, In: Proceedings of ACM SIGCOMM 2001
    [52]Rowstron A, Druschel P, Pastry: Scalable, Distributed Object Location and Routing for Large-scale Peer-to-Peer Systems. In: IFIP/ACM International Conference on Distributed Systems Platforms (Middleware), Heidelberg, Germany, 2001: 329-350
    [53]Karger D, Lehman E, Leighton F, et al., Consistent hashing and random trees: Distributed caching protocols for relieving hot spots on the World Wide Web. In: Proceedings of the 29th Annual ACM Symposium on Theory of Computing, El Paso, Texas, 1997,654-663.
    [54]Chawathe Y, Ratnasamy S, Breslau L, Make Gnutella-like P2P Systems Scalable, In: Proceedings of ACM SIGCOMM 2003
    [55]http://rfc-gnutella.sourceforge.net/src/rfc-0_6-draft.html
    [56] Cohen E, Shenker S, Replication Strategies in Unstructured Peer-to-Peer Networks, In: Proceedings of ACM SIGCOMM 2002
    [57]Crespo A, Garcia H, Routing Indices for Peer-to-Peer Systems, In: Proceedings of the 22nd International Conference on Distributed Computing Systems (ICDCS'02)
    [58]Sripanidkulchai K, Maggs B, Zhang H, Efficient Content Location Using Interest-Based Locality in Peer-to-Peer Systems. In: Proceedings of INFOCOM 2003.
    [59]Iamnitchi A, Ripeanu M, Foster I, Small-World File-Sharing Communities. In: Proceedings of INFOCOM 2004, Hong Kong, 2004.
    [60]Jovanovic M, Modeling Large-scale Peer-to-Peer Networks and a Case Study of Gnutella. Master's thesis, University of Cincinnati, 2001.
    [61] Ripeanu M, Foster I, Mapping the Gnutella Network: Macroscopic Properties of Large-Scale Peer-to-Peer Systems, In: Proceedings of the 1st International Workshop on Peer-to-Peer Systems (IPTPS'02), 2002.
    [62] Ripeanu M, Peer-to-Peer Architecture Case Study: Gnutella Network, In: Proceedings of IEEE P2P2001,2001.
    [63]Faloutsos M, Faloutsos P, Faloutsos C, On Power-Law Relationships of the Internet Topology, In: Proceedings of ACM SIGCOMM 1999.
    [64]Vaucher J, Kropf P, Babin G, Experimenting with Gnutella Communities, In: Proceedings of 4th International Conference on Distributed Communities on the Web (DCW2002), Sydney, 2002.
    [65] Adamic L, Lukose R, Huberman B, Search in Power-law Networks, Physical review 2001, 64(9).
    
    [66] Lv Q, Ratnasamy S, Shenker S, Can Heterogeneity make Gnutella Scalable? In: Proceedings of the 1st International Workshop on Peer-to-Peer Systems (IPTPS'02), 2002.
    [67] Ian Clarke, Oskar Sandberg, Brandon Wiley, Freenet: A Distributed Anonymous Information Storage and Retrieval System, In: Proceedings of International Workshop on Design Issues in Anonymity and Unobservability, 2000.
    [68] Zhang H, Goel A, Govindan R, Using the Small-World Model to Improve Freenet Performance,ACM SIGCOMM Computer Communication Review,2002,32(1):78-88.
    [69]Strogatz S,Watts D.Characteristics of Small World Networks.Nature,1998,393(6):440-442.
    [70]Huebsch R,Loo B,Shenker S,Complex Queries in DHT-based Peer-to-Peer Networks,In:Proceedings of the 1st International Workshop on Peer-to-Peer Systems(IPTPS'02),2002.
    [71]Balazinska M,Balakrishnan M,Karger D,INS/Twine:A Scalable Peer-to-Peer Architecture for Intentional Resource Discovery.In:Proceedings of International Conference on Pervasive Computing,Switzerland,2002
    [72]Winoto W,Schwartz E,Balakrishnan H,et al.,The Design and Implementation of an Intentional Naming System.In:Proceedings of 17th ACM SOSP,Kiawah Island,1999
    [73]Andrzejak A,Xu Z.Scalable,Efficient Range Queries for Grid Information Services.In:Proceedings of IEEE P2P 2002.
    [74]Schlosser M,Sintek M,Decker S,et al.,A Scalable and Ontology-Based P2P Infrastructure for Semantic Web Services,In:Proceedings of IEEE P2P2002.
    [75]Huang L,Wu Z,Pan Y,A Scalable and Effective Architecture for Grid services' Discovery,In:Proceedings of WWW 2003.
    [76]Chander A,Dawson S,Lincoln P et al..NEVRLATE:Scalable Resource Discovery.In:Proceedings of IEEE/ACM CCGrid 2002.
    [77]Li W,Xu Z,Dong F,et al.,A Grid Resource Discovery Model based on the Routing-Transferring Method,In:In:Proceedings of 3rd IEEE/ACM International Workshop on Grid Computing 2002.
    [78]李伟 徐志伟 卜冠英等,数据网格环境下一种有效的资源查找方法,计算机学报,2003,26(11):1546-1549
    [79]Weglarz J,Multicriteria Aspects of Grid Resource Management,In:Proceedings of 5th International Conference on Parallel Processing and Applied Mathematics,Czestochowa,Poland,2003.
    [80]Karbhari P,Ammar M,Dhamdhere A,Bootstrapping in Gnutella - A Preliminary Measurement Study,In:Proceedings of 5th Passive and Active Measurement Workshop,France,2004.
    [81]Andersen D,Balakrishnan H,Kaashoek M,Resilient Overlay Networks,In:Proceedings ofACM SOSP,Banff,Canada,2001.
    [82]Jelasity M,Preusb M,Steen M.Maintaining Connectivity in a Scalable and Robust Distributed Environment.In:Proceedings of 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid(CCGRID'02),Berlin,Germany,2002.
    [83]Laherrere J, Somette D, Stretched exponential distributions in Nature and Economy: Tat tails' with characteristic scales, European Physical Journals, 1998, B2:525-539
    [84] Gupta I, Birman K, Linga P, Kelips: building an efficient and stable P2P DHT through increased memory and background overhead, In: Proceedings of the 2nd International Workshop on Peer-to-Peer Systems (IPTPS'03), 2003.
    [85]Motwani R, Raghavan P. Randomized Algorithms[M]. London: Cambridge University Press, 1995
    [86]Stoica I, Morris R, Karger D, et al., Chord: A Scalable Peer-To-Peer Lookup Service for Internet Applications[R]. TR-819, MIT LCS, 2001. http://www.pdos.lcs.mit.edu/chord/papers
    [87]Demer A, Greene D, Hauser C, et al.. Epidemic Algorithm for Replicated Database Maintenance [A]. In: Proceedings of the 6th ACM Symposium on Principles of Distributed Computing, 1987,1-12.
    [88] Boris P. On Spreading a Rumor. SIAM Journal of Applied Mathematics, 1987, 47(1): 213-223.
    [89]Karp P, Schindelhauer C, Shenker S, et al.. Randomized Rumor Spreading. In: Proceedings of the 41st Annual Symposium on Foundation of Computer Science, 2000.
    [90]Freedman M, Mazieres D. Sloppy hashing and self-organize clusters[A]. In: Proceedings of the 2nd International Workshop on Peer-to-Peer Systems (IPTPS '03), Berkeley, 2003. 45-55
    [91]Byers J, Considine J, Mitzenmacher M. Simple Load Balancing in Distributed Hashing Tables. In: Proceedings of the 2nd International Workshop on Peer-to-Peer Systems (IPTPS '03), Berkeley, 2003.
    [92]Rao A, Lakshminarayanan K, Suranaet S et al.. Load Balancing in Structured P2P Systems. In: Proceedings of the 2nd International Workshop on Peer-to-Peer Systems (IPTPS '03), Berkeley, 2003.
    [93]Datta A, Hauswirth M, Aberer K. Updates in Highly Unreliable, Replicated Peer-to-Peer Systems. In: Proceedings of the ICDCS 2003.
    [94]Luo J, Eugster P, Hubaux J. Route Driven Gossip: Probabilistic Reliable Multicast in Ad Hoc Networks. In: Proceedings of 22nd Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM 2003), 2003.
    [95]Eugster P, Guerraoui R, Handurukande S. Lightweight Probabilistic Broadcast. In: Proceedings of The International Conference on Dependable Systems and Networks (DSN 2001), 2001
    [96] Kermarrec A, Massoulie L, Ganesh A. Reliable probabilistic communication in large-scale information dissemination systems. MMSR-TR-2000-105, 2000. http://research.microsoft.com/camdis/PUBLIS/Kermarrec_techreport.pdf
    [97]Dou W,Jia Y,Wang H,et al.,A P2P Approach for Global Computing.to appear In:Proceedings of the IPDPS03,New York:IEEE Press,2003.
    [98]窦文 贾焰 王怀民等,基于对端重叠网络的通用大规模计算资源共享环境的构造,计算机学报,2004,27(1):21-31
    [99]Andrzejak A,Graupner S,Kotov V,et al.,Self-Organizing Control in Planetary-Scale Computing,In:Proceedings of the 2nd International Symposium on Cluster Computing and the Grid(CCGrid 2002),New York:IEEE Press,2002
    [100]Palmer C,Steffan J,Generating Network Topologies that Obey Power Laws,In:Proceedings of the IEEE Globecom'00,San Francisco,2000.
    [101]Newman M,Strogatz SH,Watts DJ.Random Graphs with Arbitrary Distributions and Their Applications.Physical Review,2001,64(2)
    [102]Kamvar S,Schlosser M,EigenRep:reputation Management in P2P Networks,In:Proceedings of 12th International World Wide Web Conference,Budapest,2003.
    [103]Iamnitchi A,Foster I,A Peer-to-Peer Approach to Resource Discovery in Grid Environments,In:Proceedings of the 11th Symposium on High Performance Distributed Computing,New York:IEEE Press,2002
    [104]Daniel S.Bernstein,Zhengzhu Feng,et al..Adaptive Peer Selection.In:Proceedings of the 2nd International Workshop on Peer-to-Peer Systems(IPTPS '03),Heidelberg:Springer-Verlag,2003.
    [105]Dongyan X,Klara N,Duangdao W.QoS-Aware Discovery of Wide-Area Distributed Services,In:Proceedings of the 1 st International Symposium on Cluster Computing and the Grid(CCGrid 2001),New York:IEEE Press,2001.
    [106]Munindar PS,Bin Y,Mahadevan V.Community-based Service Location,Communications of the ACM,2001,44(4):49-54.
    [107]Yang,Beverly,Garcia M.Comparing Hybrid Peer-to-Peer Systems,In:Proceedings of the VLDB 2001,Roma:Morgan Kaufmann 2001.561-570.
    [108]Menascé D,Lavanya K,Probability Scalable P2P Resource Location Services.ACM SIGMETRICS Performance Evaluation Review,2002,30(2):48-58.
    [109]Khambatti M,Ryu K,Dasgupta P,Peer-to-Peer Communities:Formation and Discovery,In:Proceedings of 14th IASTED Conference on Parallel and Distributed Computing Systems(PDCS),Cambridge,Massachusetts,November 2002
    [110]Buyya R,Murshed M,GridSim:A Toolkit for the Modeling and Simulation of Distributed Resource Management and Scheduling for Grid Computing,Journal of Concurrency and Computation:Practice and Experience,2002,14(13).
    [111]Vanthournout K,Deconinck G,Belmans R.A Taxonomy for Resource Discovery,In:Proceedings of International Conference on Architecture of Computing Systems(ARCS 2004),LNCS 2981,Springer.
    [112]http://www.fightaidsathome.org
    [113]龚光红 王行仁 彭晓源等.先进分布仿真技术的发展与应用,系统仿真学报,2004,16(2):222-230
    [114]李伯虎 柴旭东 朱文海等.SBA支撑环境技术的研究,系统仿真学报,2004,16(2):181-185
    [115]朱承 刘忠 张维明等 基于信息服务社区的网格资源发现方法 计算机科学2004 Vol31 No.12

© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号

地址:北京市海淀区学院路29号 邮编:100083

电话:办公室:(+86 10)66554848;文献借阅、咨询服务、科技查新:66554700