基于网格技术的数据传输处理系统
|
摘要
本文首先介绍了网格计算的相关理论,明确了以开放网格服务架构作为网格的体系结构;然后详细介绍了网格的数据传输系统,重点讨论了Globus工具包中的数据传输模型的体系结构、GridFTP数据传输协议和组件以及可靠的文件传输客户端。数据传输的算法对于系统性能起着至关重要的作用,因此第四章讨论了对于传统的并行算法的改进,提出了新的算法模型,描述了改进的算法,并给出了测试的结果。第五章介绍了对于一般网格数据传输和处理系统的需求分析、总体框架设计、模块设计。主要介绍了系统编程语言的选择、系统的运行环境、Globus ToolKit工具包以及与网格数据传输和处理密切相关的数据接口模块的实现。
At first, the thesis introduces theory about grid computation and regards Open Grid Service Architecture as system struct of grid system. Then the thesis introduces Grid Data Transfer System, viewing system struct of data transfer model on Globus toolkit, GridFTP data transfer protocol module and Reliable File Transfer client as emphases. Arithmetic of data transfer is important to system performance so the betterment of traditional parallel arithmetic is discussed in the fourth chapter. In this chapter, the model of new arithmetic is brought forward, better arithmetic is described and the result of test is educed. In the fifth chapter there is requirements analysis of general Grid-based Data Transfer and Process System, collectivity frame design and module design. In this chapter there is the programming language choice of this system, system running environment, Globus ToolKit and the implement of data interface module concerning grid data transfer and process.
At first, the thesis introduces theory about grid computation and regards Open Grid Service Architecture as system struct of grid system. Then the thesis introduces Grid Data Transfer System, viewing system struct of data transfer model on Globus toolkit, GridFTP data transfer protocol module and Reliable File Transfer client as emphases. Arithmetic of data transfer is important to system performance so the betterment of traditional parallel arithmetic is discussed in the fourth chapter. In this chapter, the model of new arithmetic is brought forward, better arithmetic is described and the result of test is educed. In the fifth chapter there is requirements analysis of general Grid-based Data Transfer and Process System, collectivity frame design and module design. In this chapter there is the programming language choice of this system, system running environment, Globus ToolKit and the implement of data interface module concerning grid data transfer and process.
引文
1 I.Foster, C.Kesselman, S.Tuecke. The Anatomy of the Grid: Enabling Scalable Virtual Organizations[J]. International J.Supercomputer Application,2001,(15):3.
2 Cluster.Computing and the Grid[J]. First IEEE/ACM International Symposium on ,2001:6-7.
3 I.Foster,C.Kesselman. Globus:A Metacomputing Infrastructure Toolkit. International Journal of Supercomputer Applications,1997,11(2):115-128.
4都志辉,李三立,陈渝,等.网格计算及其原型实现研究[J].计算机科学, 2002,29(8): l-5.
5王育峰,杨寿保,赵晓春,等.Research on security architecture and security policy of grid system[R]. Accepted by GCC2002 Workshop, 2002.
6姜国庆黄永忠牛向华.网格环境下数据传输模型的研究.微计算机信息,2005,21(10-3)
7都志辉,等.网格计算[M].北京:清华大学出版社,2002.
8 BYERS J W,LUBY M,MITZENMACHER M. Accessing multiple mirror sites in parallel:Using Tornado codes to speed up downloads[J].Proc IEEEINFOCOM,2003,1:275-283.
9 MAXEMCHUKNF.Dispersityrouting in store-and-forward network[D]. Philadelphia: UniversityofPennsylvania, 2001.
10 KRISHNAMURTHY B,MOGUL J,KIRSTOLD.Key differences between HTTP1.0 and HTTP1.1 [A]. The 8th International World Wide Web Conference[C].Canada:[s n],2001.372-407.
11刘军万,谭骏珊,吴敏.分布式实时系统中动态负载共享新策略[J].计算机工程,2002,9(5):76-77,91.
12商爱民,陆松年.快速广播协议的改进[J].计算机工程,2002,9(1):146-148.
13 Ben Kao and Hector Garcia-Molina. An overview of real-time database systems R.Tech.Report of Primceton University, Stanford University,1994(9):34-37.
14 (美)Ian Foster,Carl Kesselman著;金海,袁平鹏,石柯译,网格计算(第二版)[M],北京:电子工业2004:224-245
15 Thulasidasan S, Wu-chun Feng, Gardner M.K, optimizing GridFTP through dynamic right-sizing, Proceedings of the 12th IEEE International Symposium on High Performance Distributed Computing, 2003:1082-8907.
16 Ann Chervenak, Carl Kesselman, Scott Koranda, Brian Moe,Robert Schuler, Design and Implementation of a Data Replication Service Based on the Lightweight Data Replicator System:3-5.
17 Ann Chervenak, et al., Giggle: A Framework for Constructing Scalable Replica Location Services, in Proc. Of SC 2002.
18 H.Stockinger, F.Dono, E.Laure, S.Muzzafar etal. Grid Data Management in action:Experience in Running and Supporting Data Management Services in the EU DataGrid Project computing in high energy physics. 28, March.2003.
19 I.Foster and C.Kesselman.The Grid:Blueprint for a New Computing Infrastructure.San Fransisco,CA:Morgan Kaufmann,1999.
20 W.Allcock, J. Bresnahan, R. Kettimuthu, M. Link, C. Dumitrescu, I. Raicu, I. Foster, The Globus Striped GridFTP Framework and Server, SC'05, ACM Press, 2005:3-5.
21 Divyakant Agrawal, Amr El Abbadi, and R. C. Steike. Epidemic algorithms in replicated databases. In 16th ACM Symp. on Princ. of Database Systems (PODS), pages 161–172, Tucson, AZ, May 2002.
22 Y.Zhou, L.Iftode, and K.Li, Performance Evaluation of Two Home-Based Lazy Release Consistency Protocols for Shared Virtual Memory Systems, Proc. of 2nd USNIX Symp.Operating Systems Design and Implementation (OSDI-2), 2004.
23 Molesky L. D.Ramamritham K.Efficient Locking for Shared Memory Database Systems R.ComputerScience Technical Report 94-US: University of Massachusetts Dept. ofcomputer Science, 2004(8):14-20.
24 Foster I,Kesselman C,Nick J M,et al.Grid Services for Distributed System Integration. IEEE Computer,2002;June:37-46.
25孙晶,李晓明.基于ODBC技术的数据导换在异构电力信息系统中的应用[J].电网技术.2004,28(4) 64-68
26王娟娟,李晓明,张莲梅,等.基于Web的多异构电力信息集成系统的开发与应用[J].电力科学与工程.2003,4:58～61
27沈沉,王继业.网格技术及其在电力系统中的应用[J].中国电力.2004,37(5):77～80
28巴拉赫,兰宝(著),车皓阳(译).UML面向对象建模与设计(第二版)[M].北京:人民邮电出版社,2006
29黄鹂,基于网格的数据管理系统的研究与实现. [硕士学位论文]南京航空航天大学2006
30 (美)Joshy Joseph, (美)Craig Fellenstein著;战晓苏,张少华译,网格计算[M],北京,清华大学出版社,2005.
31袁鹏飞著.Oracle数据库8i高级应用开发技术,人民邮电出版社,2001.7
32王珊,张坤龙,网格环境下的数据库系统,计算机应用,Vol.24.No.10 Oct.2004
33郭盈发,张红娟,数据库原理[M],西安-西安电子科技大学出版社,2002.2:120–137
34 Paul Dubcis, MsSQL网络数据库指南[M].北京:机械工业出版社,2000,6
35徐志伟,冯百明,李伟.网格计算技术[M].北京:电子工业出版社,2004
36 Osamu Tatebe, Hirotaka Ogawa, Yuetsu Kodama, The Second Trans-Pacific Grid Datafarm Testbed and Experiments for SC2003,Proceedings of 2004 International Symposium on Applications and the Internet-Workshops (SAINT 2004 Workshops), 26-30 January 2004, Tokyo,Japan
37 Shoji Ogura, Satoshi Matsuoka, Hidemoto Nakada, Evaluation of the inter-cluster data transfer on Grid environment, Proceedings of the 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid 2003), 374-381, 2003
38 Florian Schintke, A Framework for Self-Optimizing Grids Using P2P Components[J]. Proceedings of the 14th International Workshop on Database and Expert Systems Applications(DEXA’03) 200339
39季一木,王汝传.基于OGSA/OGSI和Eclipse的网格计算平台的设计[J].电子工程师.2005,31(1):64-65
40 GAllen,T.Dramlitsch and B.Toonen.Supporting Efficient Execution in Heterogeneous Distributed Computing Environments with Cactus and Globus. Proceedings of the SC 2001,November 10-16,2001
41 David De Roure,J A Hendler.E-Science: The grid and the semantic Web. IEEE Intelligent Systems, 2004,19(1):65-71
42黄允中,顾志松,张世永.网格技术框架的探讨和研究[J].计算机工程,2003,29 (13):13 3-1
43宋智礼,余生晨,刘凡美.应用网格技术实现校园网资源共享模型的研究[J].北方工业大学学报,2004,16(1):12-16
44桂小林,钱德沛.基于Internet的网格计算模型研究[J].西安交通大学学报,2001,35(10)1008-1011
45查义国,徐小岩,张毓森在Web上实现基于角色的访问控制[J].计算机研究与发展,2002,39(3):257-263
2 Cluster.Computing and the Grid[J]. First IEEE/ACM International Symposium on ,2001:6-7.
3 I.Foster,C.Kesselman. Globus:A Metacomputing Infrastructure Toolkit. International Journal of Supercomputer Applications,1997,11(2):115-128.
4都志辉,李三立,陈渝,等.网格计算及其原型实现研究[J].计算机科学, 2002,29(8): l-5.
5王育峰,杨寿保,赵晓春,等.Research on security architecture and security policy of grid system[R]. Accepted by GCC2002 Workshop, 2002.
6姜国庆黄永忠牛向华.网格环境下数据传输模型的研究.微计算机信息,2005,21(10-3)
7都志辉,等.网格计算[M].北京:清华大学出版社,2002.
8 BYERS J W,LUBY M,MITZENMACHER M. Accessing multiple mirror sites in parallel:Using Tornado codes to speed up downloads[J].Proc IEEEINFOCOM,2003,1:275-283.
9 MAXEMCHUKNF.Dispersityrouting in store-and-forward network[D]. Philadelphia: UniversityofPennsylvania, 2001.
10 KRISHNAMURTHY B,MOGUL J,KIRSTOLD.Key differences between HTTP1.0 and HTTP1.1 [A]. The 8th International World Wide Web Conference[C].Canada:[s n],2001.372-407.
11刘军万,谭骏珊,吴敏.分布式实时系统中动态负载共享新策略[J].计算机工程,2002,9(5):76-77,91.
12商爱民,陆松年.快速广播协议的改进[J].计算机工程,2002,9(1):146-148.
13 Ben Kao and Hector Garcia-Molina. An overview of real-time database systems R.Tech.Report of Primceton University, Stanford University,1994(9):34-37.
14 (美)Ian Foster,Carl Kesselman著;金海,袁平鹏,石柯译,网格计算(第二版)[M],北京:电子工业2004:224-245
15 Thulasidasan S, Wu-chun Feng, Gardner M.K, optimizing GridFTP through dynamic right-sizing, Proceedings of the 12th IEEE International Symposium on High Performance Distributed Computing, 2003:1082-8907.
16 Ann Chervenak, Carl Kesselman, Scott Koranda, Brian Moe,Robert Schuler, Design and Implementation of a Data Replication Service Based on the Lightweight Data Replicator System:3-5.
17 Ann Chervenak, et al., Giggle: A Framework for Constructing Scalable Replica Location Services, in Proc. Of SC 2002.
18 H.Stockinger, F.Dono, E.Laure, S.Muzzafar etal. Grid Data Management in action:Experience in Running and Supporting Data Management Services in the EU DataGrid Project computing in high energy physics. 28, March.2003.
19 I.Foster and C.Kesselman.The Grid:Blueprint for a New Computing Infrastructure.San Fransisco,CA:Morgan Kaufmann,1999.
20 W.Allcock, J. Bresnahan, R. Kettimuthu, M. Link, C. Dumitrescu, I. Raicu, I. Foster, The Globus Striped GridFTP Framework and Server, SC'05, ACM Press, 2005:3-5.
21 Divyakant Agrawal, Amr El Abbadi, and R. C. Steike. Epidemic algorithms in replicated databases. In 16th ACM Symp. on Princ. of Database Systems (PODS), pages 161–172, Tucson, AZ, May 2002.
22 Y.Zhou, L.Iftode, and K.Li, Performance Evaluation of Two Home-Based Lazy Release Consistency Protocols for Shared Virtual Memory Systems, Proc. of 2nd USNIX Symp.Operating Systems Design and Implementation (OSDI-2), 2004.
23 Molesky L. D.Ramamritham K.Efficient Locking for Shared Memory Database Systems R.ComputerScience Technical Report 94-US: University of Massachusetts Dept. ofcomputer Science, 2004(8):14-20.
24 Foster I,Kesselman C,Nick J M,et al.Grid Services for Distributed System Integration. IEEE Computer,2002;June:37-46.
25孙晶,李晓明.基于ODBC技术的数据导换在异构电力信息系统中的应用[J].电网技术.2004,28(4) 64-68
26王娟娟,李晓明,张莲梅,等.基于Web的多异构电力信息集成系统的开发与应用[J].电力科学与工程.2003,4:58～61
27沈沉,王继业.网格技术及其在电力系统中的应用[J].中国电力.2004,37(5):77～80
28巴拉赫,兰宝(著),车皓阳(译).UML面向对象建模与设计(第二版)[M].北京:人民邮电出版社,2006
29黄鹂,基于网格的数据管理系统的研究与实现. [硕士学位论文]南京航空航天大学2006
30 (美)Joshy Joseph, (美)Craig Fellenstein著;战晓苏,张少华译,网格计算[M],北京,清华大学出版社,2005.
31袁鹏飞著.Oracle数据库8i高级应用开发技术,人民邮电出版社,2001.7
32王珊,张坤龙,网格环境下的数据库系统,计算机应用,Vol.24.No.10 Oct.2004
33郭盈发,张红娟,数据库原理[M],西安-西安电子科技大学出版社,2002.2:120–137
34 Paul Dubcis, MsSQL网络数据库指南[M].北京:机械工业出版社,2000,6
35徐志伟,冯百明,李伟.网格计算技术[M].北京:电子工业出版社,2004
36 Osamu Tatebe, Hirotaka Ogawa, Yuetsu Kodama, The Second Trans-Pacific Grid Datafarm Testbed and Experiments for SC2003,Proceedings of 2004 International Symposium on Applications and the Internet-Workshops (SAINT 2004 Workshops), 26-30 January 2004, Tokyo,Japan
37 Shoji Ogura, Satoshi Matsuoka, Hidemoto Nakada, Evaluation of the inter-cluster data transfer on Grid environment, Proceedings of the 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid 2003), 374-381, 2003
38 Florian Schintke, A Framework for Self-Optimizing Grids Using P2P Components[J]. Proceedings of the 14th International Workshop on Database and Expert Systems Applications(DEXA’03) 200339
39季一木,王汝传.基于OGSA/OGSI和Eclipse的网格计算平台的设计[J].电子工程师.2005,31(1):64-65
40 GAllen,T.Dramlitsch and B.Toonen.Supporting Efficient Execution in Heterogeneous Distributed Computing Environments with Cactus and Globus. Proceedings of the SC 2001,November 10-16,2001
41 David De Roure,J A Hendler.E-Science: The grid and the semantic Web. IEEE Intelligent Systems, 2004,19(1):65-71
42黄允中,顾志松,张世永.网格技术框架的探讨和研究[J].计算机工程,2003,29 (13):13 3-1
43宋智礼,余生晨,刘凡美.应用网格技术实现校园网资源共享模型的研究[J].北方工业大学学报,2004,16(1):12-16
44桂小林,钱德沛.基于Internet的网格计算模型研究[J].西安交通大学学报,2001,35(10)1008-1011
45查义国,徐小岩,张毓森在Web上实现基于角色的访问控制[J].计算机研究与发展,2002,39(3):257-263