网格环境下资源管理机制的研究
摘要
网格是构筑在互联网基础之上的新兴分布式计算技术。它通过整合分布在各地的计算、存储和通信资源,为动态变化的虚拟组织的成员提供更为广泛的资源共享。网格是通过集中分散的资源来完成大的作业任务,资源的共享是一种集中资源的手段。网格通过对资源的高度抽象将网络上分散的资源以用户可见的接口提供给用户使用。网格的动态性和多样性使网格对资源的管理需要动态的考虑。
本文首先对网格的研究现状、基本概念、体系结构做了阐述,并对MDS,RLS等关键技术进行了深入的研究。同时结合油田数据管理方面的需要,提出了基于网格的分布式数据管理平台模型;其次,重点研究了模型中涉及的两项关键技术——网格资源监测和发现以及副本定位服务,在对资源监测和发现模型的研究过程中,提出了一种在客户端实现对网格资源监测的方法并实现了一个MDS客户端查询原型;最后,基于所提出的模型设计了分布式数据管理系统,并对原型系统进行了部分实现。
结果表明,分布式数据管理的模型可以在各个部件的协同工作下,将具有副本的大数据量的数据的访问问题很好的管理起来。实现了对网格环境下的副本资源的管理。
Grid is a new emerging technology, which is based on Internet. It provides more widespread resources sharing for the dynamic change VO's members, by integrating computation resources, the memory resources and the correspondence resources, which are distributed on network. The grid completes the big work task through collecting distributed resources, and resources sharing are one method of collecting resources. The grid provides the visible interface to the user by highly abstracting the resource, which is distributed on network. Grid is dynamic and diverse, and that needs the dynamic consideration in grid resources management.
Firstly, on the basis of the studies of grid architecture, MDS, RLS, and combined with the requirement of data management of Oil Field, this thesis advances a platform model of distributed data management based on grid. Secondly, it focuses on two points of key technologies involved in the model. One is grid resources monitoring and discovery, the other is Replica Location Services. In resources monitor and discovery model research process, a method of aggregating information based on MDS is advanced, and a prototype of MDS query client is implemented. Finally, based on the model which proposed has designed the distributed data management system, and has implemented the partial functions in the prototype system.
It is proved that the distributed data management model by the cooperation of each part could have the problem of the great data accessing managed. The replica resources management has been completed in grid.
本文首先对网格的研究现状、基本概念、体系结构做了阐述,并对MDS,RLS等关键技术进行了深入的研究。同时结合油田数据管理方面的需要,提出了基于网格的分布式数据管理平台模型;其次,重点研究了模型中涉及的两项关键技术——网格资源监测和发现以及副本定位服务,在对资源监测和发现模型的研究过程中,提出了一种在客户端实现对网格资源监测的方法并实现了一个MDS客户端查询原型;最后,基于所提出的模型设计了分布式数据管理系统,并对原型系统进行了部分实现。
结果表明,分布式数据管理的模型可以在各个部件的协同工作下,将具有副本的大数据量的数据的访问问题很好的管理起来。实现了对网格环境下的副本资源的管理。
Grid is a new emerging technology, which is based on Internet. It provides more widespread resources sharing for the dynamic change VO's members, by integrating computation resources, the memory resources and the correspondence resources, which are distributed on network. The grid completes the big work task through collecting distributed resources, and resources sharing are one method of collecting resources. The grid provides the visible interface to the user by highly abstracting the resource, which is distributed on network. Grid is dynamic and diverse, and that needs the dynamic consideration in grid resources management.
Firstly, on the basis of the studies of grid architecture, MDS, RLS, and combined with the requirement of data management of Oil Field, this thesis advances a platform model of distributed data management based on grid. Secondly, it focuses on two points of key technologies involved in the model. One is grid resources monitoring and discovery, the other is Replica Location Services. In resources monitor and discovery model research process, a method of aggregating information based on MDS is advanced, and a prototype of MDS query client is implemented. Finally, based on the model which proposed has designed the distributed data management system, and has implemented the partial functions in the prototype system.
It is proved that the distributed data management model by the cooperation of each part could have the problem of the great data accessing managed. The replica resources management has been completed in grid.
引文
[1] 张丽宁.网格技术的发展与研究现状[J].信息化与网络建设.2005,(10).
[2] 徐钊,刘志强.第三代 Internet-网格及其关键技术研究[J].计算机与网络,2005,(3):80.
[3] 金海,郑纬民.网格计算专刊前言[J].软件学报,2006,17(11):2221-2223.
[4] 中国科学院计算机网络信息中心.欧盟 FP6 计划 BRIDGE 项目启动[DB/OL]. http://www.cnic.ac.cn/science/communion/200701230007.html
[5] 网格与服务计算研究中心[DB/OL].http://vega.ict.ac.cn/
[6] The globus alliance [DB/OL].http://www.globus.org
[7] Ian Foster,Carl Kesselman. The Globus Project: A Status Report. In Proc. Heterogeneous Computing Workshop[J]. IEEE Computer Society Press, 1998: 4-18.
[8] Ian Forster and Carl Kesselman. Globus: A Meta-computing Infrastructure Toolkit[J]. Supercomputer Applications, 1996: 438-450.
[9] Ian Foster, Carl Kesselman, Steven Tuecke. The Anatomy of the Grid: Enabling Scalable Virtual Organizations [J]. International Journal Supercomputer Applications, 2001,15(3): 200-222.
[10] 邹锐.网格资源管理的模型研究[D].华中科技大学,2004.
[11] 都志辉,陈渝,刘鹏.网格计算[M].清华大学出版社,2002:9-12.
[12] Ian Foster,Carl Kesselman. The Grid Blueprint for a New Computing Infrastructure (Second Edition)[M].机械工业出版社.2005:47-52.
[13] Towards Open Grid Services Architecture[DB/OL].http://www.globus.org/ogsa/
[14] The WS-Resource Framework [DB/OL].http://www.globus.org/wsrf/
[15] 李珍香,谢连山.网格体系结构及其发展研究[J].计算机工程.2005,31.
[16] 都 志 辉 , 陈 渝 , 刘 鹏 . 以服务为中心的网格体系结构 OGSA[J]. 计算机科学,2003,30(7):26-29.
[17] Web Services Description Language (WSDL) Version 2.0 Part 0: Primer[DB/OL]. http://www.w3.org/TR/2006/CR-wsdl20-primer-20060327/.2006.
[18] 赵念强 , 鞠时光 . 网格计算及网格体系结构研究综述 [J]. 计算机工程与设计.2006,27(5):729-730.
[19] S. Tuecke, K. Czajkowski, I. Foster, J. Frey, S. Graham, C. Kesselman, T. Maquire, T. Sandholm, D. Snelling, P. Vanderbilt. Open Grid Services Infrastructure (OGSI) ersion 1.0. June 27, 2003
[20] Karl Czajkowski, Donald F Ferguson, Ian Foster, et al. The WS-Resource Framework[DB/OL]. http://www.ibm.com/developerworks/library/ws-resource/ws-wsrf.pdf. 2004.
[21] 徐志伟,冯百名,李伟.网格计算技术[M].电子工业出版社,2004:94-97.
[22] 蒋兴胜.对等计算技术在网格资源管理中的应用研究与实现[D].2005.
[23] 何琨,赵勇.网格资源管理与调度研究综述[J]. 武汉理工大学学报.2005,27(4).
[24] 樊宁.网格的资源管理和信息管理概述[DB/OL].http://www.ibm.com/developerwor ks/cn/grid/gr-fann1/.2006.
[25] 王庆荣,卫祥.网格资源管理系统模型研究[J].微机算计信息.2006,22,(1-3).
[26] Marc Farley,孙功星,蒋文保,范勇译. SAN 存储区域网络[M].北京:机械工业出版社,2002
[27] Information Services (MDS):Key Concepts[DB/OL]. http://www.globus.org/toolkit/ docs/4.0/info/key-index.html
[28] GT4 RLS[DB/OL]. http://www.globus.org/toolkit/docs/4.0/data/rls/
[29] GT4 GridFTP[DB/OL]. http://www.globus.org/toolkit/docs/4.0/data/gridftp/
[30] Steve Graham, Bryan Murray, eds. Web Services Base Notification 1.2 (WS-BaseNotifi- cation), OASIS Working Draft wsn-WS-BaseNotification-1.2-draft–03 [DB/OL], June 2004, http://docs.oasis-open.org/wsn/2004/06/wsn-WS-BaseNotification -1.2- draft -03.pdf
[31] GT 4.0: Information Services: Index[DB/OL].http://www.globus.org/toolkit/docs/4.0 /info/index/
[32] GT 4.0 WS MDS Trigger Service[DB/OL].http://www.globus.org/toolkit/docs/4.0/info /triger/
[33] Deploying WS MDS in a Virtual Organization[DB/OL].http://www.globus.org/toolkit/ docs/4.0/info/key/deployment_overview.html
[34] Tom Maguire, David Snelling, eds. Web Services Service Group 1.2 (WS-ServiceGroup) [R].OASIS Working Draft wsrf-WS-ServiceGroup-1.2-draft-02. http://docs.oasis-open.org/wsrf/2004/06/wsrf-WS-ServiceGroup-1.2-draft-02.pdf. 2004.
[35] UDDI Standard[DB/OL].http://www.uddi.org
[36] GT 4.0: Information Services: Aggregator Framework[DB/OL].http://www.globus.org/ toolkit/docs/4.0/info/aggregator/
[37] Ganglia[DB/OL].http://ganglia.sourceforge.net/
[38] Glue Computing Element Schema version 1.1 Final [DB/OL]. http://www.cnaf.infn.it/ ~andreozzi /datatag/glue/v11/CE/GlueCE_DOC_V_1_1.htm
[39] Java COG Kit [DB/OL]. http://wiki.cogkit.org/index.php/Main_Page
[40] Java WS Core [DB/OL]. http://www.globus.org/toolkit/docs/4.0/common/javawscore/
[41] Ann Chervenak, Ewa Deelman, Ian Foster et al. Giggle: A Framework for Constructing Scalable Replica Location Services[J].IEEE.2002.
[42] Ann Chervenak, Robert Schuler, Carl Kesselman et al. Wide Area Data Replication for Scientific Collaborations[J].Inderscience Enterprises Ltd.2006.
[43] Ann L. Chervenak, Naveen Palavalli, Shishir Bharathi Performance and Scalability of a Replica Location Service[R]. High Performance Distributed Computing (HPDC-13) Conference, 2004.
[44] Borja Sotomayor,The Globus Toolkit 4 Programmer’s Tutorial. University of Chicago Department of computer Science.15-17.
[45] Jennifer M.Schopf, Mike D’Arcy, Neill Miller, et al. Monitoring and Discovery in a Web Services Framework: Functionality and Performance of the Globus Toolkit’s MDS4[R]. Argonne National Laboratory Technical Report.2005.
[46] MDS4 and Project Deployments[DB/OL].http://www.globus.org/toolkit/mds2005.
[47] Man Yuan,Changying Sun, Pengfei Li,et al. Improved resource management model based on MDS[C]. Proceedings of SPIE-The International Society for Optical Engineering 6022,2005:1040-1049
[48] Xuehai Zhang, Jeffrey L. Freschl, and Jennifer M. Schopf, A Scalability Analysis of Three Monitoring and Information Systems: MDS2, R-GMA, and Hawkeye[R]. http://www.mcs.anl.gov/~jms/jmspubs.html.2005.
[49] X. Zhang and J. M. Schopf, Performance Analysis of the Globus Toolkit Monitoring and Discovery Service, MDS2[C]. Proceedings of IEEE IPCCC International Workshop on Middleware Performance.2004.
[50] Kevin Flasch.Lightweight Data Replicator (LDR) Administrator’s Guide[R].2005.
[51] Sudharshan Vazhkudai,Jennifer M. Schopf. Using Disk Throughput Data in Predictions of End-to-End Grid Data Transfers[R].2004.
[52] Ioan Raicu. A Performance Study of the Globus Toolkit(R) and Grid Services via Diperf, an Automated Distributed Performance Testing Framework[D].2005.
[2] 徐钊,刘志强.第三代 Internet-网格及其关键技术研究[J].计算机与网络,2005,(3):80.
[3] 金海,郑纬民.网格计算专刊前言[J].软件学报,2006,17(11):2221-2223.
[4] 中国科学院计算机网络信息中心.欧盟 FP6 计划 BRIDGE 项目启动[DB/OL]. http://www.cnic.ac.cn/science/communion/200701230007.html
[5] 网格与服务计算研究中心[DB/OL].http://vega.ict.ac.cn/
[6] The globus alliance [DB/OL].http://www.globus.org
[7] Ian Foster,Carl Kesselman. The Globus Project: A Status Report. In Proc. Heterogeneous Computing Workshop[J]. IEEE Computer Society Press, 1998: 4-18.
[8] Ian Forster and Carl Kesselman. Globus: A Meta-computing Infrastructure Toolkit[J]. Supercomputer Applications, 1996: 438-450.
[9] Ian Foster, Carl Kesselman, Steven Tuecke. The Anatomy of the Grid: Enabling Scalable Virtual Organizations [J]. International Journal Supercomputer Applications, 2001,15(3): 200-222.
[10] 邹锐.网格资源管理的模型研究[D].华中科技大学,2004.
[11] 都志辉,陈渝,刘鹏.网格计算[M].清华大学出版社,2002:9-12.
[12] Ian Foster,Carl Kesselman. The Grid Blueprint for a New Computing Infrastructure (Second Edition)[M].机械工业出版社.2005:47-52.
[13] Towards Open Grid Services Architecture[DB/OL].http://www.globus.org/ogsa/
[14] The WS-Resource Framework [DB/OL].http://www.globus.org/wsrf/
[15] 李珍香,谢连山.网格体系结构及其发展研究[J].计算机工程.2005,31.
[16] 都 志 辉 , 陈 渝 , 刘 鹏 . 以服务为中心的网格体系结构 OGSA[J]. 计算机科学,2003,30(7):26-29.
[17] Web Services Description Language (WSDL) Version 2.0 Part 0: Primer[DB/OL]. http://www.w3.org/TR/2006/CR-wsdl20-primer-20060327/.2006.
[18] 赵念强 , 鞠时光 . 网格计算及网格体系结构研究综述 [J]. 计算机工程与设计.2006,27(5):729-730.
[19] S. Tuecke, K. Czajkowski, I. Foster, J. Frey, S. Graham, C. Kesselman, T. Maquire, T. Sandholm, D. Snelling, P. Vanderbilt. Open Grid Services Infrastructure (OGSI) ersion 1.0. June 27, 2003
[20] Karl Czajkowski, Donald F Ferguson, Ian Foster, et al. The WS-Resource Framework[DB/OL]. http://www.ibm.com/developerworks/library/ws-resource/ws-wsrf.pdf. 2004.
[21] 徐志伟,冯百名,李伟.网格计算技术[M].电子工业出版社,2004:94-97.
[22] 蒋兴胜.对等计算技术在网格资源管理中的应用研究与实现[D].2005.
[23] 何琨,赵勇.网格资源管理与调度研究综述[J]. 武汉理工大学学报.2005,27(4).
[24] 樊宁.网格的资源管理和信息管理概述[DB/OL].http://www.ibm.com/developerwor ks/cn/grid/gr-fann1/.2006.
[25] 王庆荣,卫祥.网格资源管理系统模型研究[J].微机算计信息.2006,22,(1-3).
[26] Marc Farley,孙功星,蒋文保,范勇译. SAN 存储区域网络[M].北京:机械工业出版社,2002
[27] Information Services (MDS):Key Concepts[DB/OL]. http://www.globus.org/toolkit/ docs/4.0/info/key-index.html
[28] GT4 RLS[DB/OL]. http://www.globus.org/toolkit/docs/4.0/data/rls/
[29] GT4 GridFTP[DB/OL]. http://www.globus.org/toolkit/docs/4.0/data/gridftp/
[30] Steve Graham, Bryan Murray, eds. Web Services Base Notification 1.2 (WS-BaseNotifi- cation), OASIS Working Draft wsn-WS-BaseNotification-1.2-draft–03 [DB/OL], June 2004, http://docs.oasis-open.org/wsn/2004/06/wsn-WS-BaseNotification -1.2- draft -03.pdf
[31] GT 4.0: Information Services: Index[DB/OL].http://www.globus.org/toolkit/docs/4.0 /info/index/
[32] GT 4.0 WS MDS Trigger Service[DB/OL].http://www.globus.org/toolkit/docs/4.0/info /triger/
[33] Deploying WS MDS in a Virtual Organization[DB/OL].http://www.globus.org/toolkit/ docs/4.0/info/key/deployment_overview.html
[34] Tom Maguire, David Snelling, eds. Web Services Service Group 1.2 (WS-ServiceGroup) [R].OASIS Working Draft wsrf-WS-ServiceGroup-1.2-draft-02. http://docs.oasis-open.org/wsrf/2004/06/wsrf-WS-ServiceGroup-1.2-draft-02.pdf. 2004.
[35] UDDI Standard[DB/OL].http://www.uddi.org
[36] GT 4.0: Information Services: Aggregator Framework[DB/OL].http://www.globus.org/ toolkit/docs/4.0/info/aggregator/
[37] Ganglia[DB/OL].http://ganglia.sourceforge.net/
[38] Glue Computing Element Schema version 1.1 Final [DB/OL]. http://www.cnaf.infn.it/ ~andreozzi /datatag/glue/v11/CE/GlueCE_DOC_V_1_1.htm
[39] Java COG Kit [DB/OL]. http://wiki.cogkit.org/index.php/Main_Page
[40] Java WS Core [DB/OL]. http://www.globus.org/toolkit/docs/4.0/common/javawscore/
[41] Ann Chervenak, Ewa Deelman, Ian Foster et al. Giggle: A Framework for Constructing Scalable Replica Location Services[J].IEEE.2002.
[42] Ann Chervenak, Robert Schuler, Carl Kesselman et al. Wide Area Data Replication for Scientific Collaborations[J].Inderscience Enterprises Ltd.2006.
[43] Ann L. Chervenak, Naveen Palavalli, Shishir Bharathi Performance and Scalability of a Replica Location Service[R]. High Performance Distributed Computing (HPDC-13) Conference, 2004.
[44] Borja Sotomayor,The Globus Toolkit 4 Programmer’s Tutorial. University of Chicago Department of computer Science.15-17.
[45] Jennifer M.Schopf, Mike D’Arcy, Neill Miller, et al. Monitoring and Discovery in a Web Services Framework: Functionality and Performance of the Globus Toolkit’s MDS4[R]. Argonne National Laboratory Technical Report.2005.
[46] MDS4 and Project Deployments[DB/OL].http://www.globus.org/toolkit/mds2005.
[47] Man Yuan,Changying Sun, Pengfei Li,et al. Improved resource management model based on MDS[C]. Proceedings of SPIE-The International Society for Optical Engineering 6022,2005:1040-1049
[48] Xuehai Zhang, Jeffrey L. Freschl, and Jennifer M. Schopf, A Scalability Analysis of Three Monitoring and Information Systems: MDS2, R-GMA, and Hawkeye[R]. http://www.mcs.anl.gov/~jms/jmspubs.html.2005.
[49] X. Zhang and J. M. Schopf, Performance Analysis of the Globus Toolkit Monitoring and Discovery Service, MDS2[C]. Proceedings of IEEE IPCCC International Workshop on Middleware Performance.2004.
[50] Kevin Flasch.Lightweight Data Replicator (LDR) Administrator’s Guide[R].2005.
[51] Sudharshan Vazhkudai,Jennifer M. Schopf. Using Disk Throughput Data in Predictions of End-to-End Grid Data Transfers[R].2004.
[52] Ioan Raicu. A Performance Study of the Globus Toolkit(R) and Grid Services via Diperf, an Automated Distributed Performance Testing Framework[D].2005.