摘要
随着网格技术和计算环境的发展,网格计算正从科学计算领域推向广阔的企业应用领域。数据网格中的网格数据库技术是企业应用中的一个重要研究课题。
本文以一个企业的实际应用为背景,在OGSA-DAI4.0体系结构的基础上,开发设计了基于发布/订阅的网格数据库实时复制系统。系统从总体构架上采用了一种分片处理技术,通过把复制任务拆分成数据发布、数据监控、数据分发、数据导入等不同的子任务,并分配在不同的服务器上并行执行,有效地减少了数据访问和计算时间,达到了异构系统间实时数据加工复制的目的。
系统的关键技术有:1)基于事务的分片数据更新机制;2)基于数据库的缓存机制;3)基于存储过程的分片计算技术;4)基于时间戳、启动阀值和信息通知的数据正确性保障机制;5)数据资源虚拟化机制;6)基于OGSA-DAI的数据访问、集成和传输机制;7)基于GT3信息通知的网格服务实时监控机制。
经测试基于发布/订阅的复制系统比基于Oracle数据链路的Pull方式的数据复制系统具有更高的效率和更好的实时性,并满足了企业实际应用需要。
With the development of grid technology and computing environment, grid computing is being pushed into the area of enterprise application from the area of science computing. It is an important area in the enterprise applications, the technology of grid database in the data grid.
According to one practical application of an enterprise, we design the real-time data replicate system of grid database, which bases on OGSA-DAI4.0 and publish/subscribe model. We reduce the time of data access and compute by dividing the replicate task into subtasks,such as data publish, data monitor, data distribute and data update, then put them into different servers, so these subtasks can run parallel. In a word, the system uses the technology of splitting process.
The key technologies include: 1) data-partition updating mechanism based on transaction; 2) cache mechanism based on database; 3) splitting compute technology based on stored procedure; 4) correct guarantee mechanism based on timestamp,threshold and notification; 5) data resource virtualization mechanism; 6) data access integration and transmit mechanism based on OGSA-DAI; 7) real-time monitor mechanism based on notification of GT3.
Test results demonstrate that the system that bases on the Publish/Subscribe has higher efficiency and good real time property against the pull-method system that bases on the Oracle data linkage, and satisfies the requires of the enterprise.
引文
1. Foster, C Kesselman. The Anatomy of the Grid: Enabling Scalable Virtual Organizations. International J. Supercomputer Application, 2001,15(3)
2. Foster, C Kesselman. The Physiology of the Grid: An Open Grid Services Architecture for Distributed System Integration. January 2002
3. 王珊,张坤 网格环境下的数据库系统[J] 计算机应用 2004(10):1-3
4. 刘建新,阎保山 OGSA-DAI 体系结构及其关键技术研究[J] 计算机应用 2004.11
5. Raman V, Narang I, Crone C, etc. “Services for Data Access and Processing on Grids”,GFD-1.14, DAIS Working Group, http://www.gridforum.org
6. 何杨平,韩海雯,沈波 一种基于数据库复制技术的分布式联机方案[J] 微机发展 2004.7
7. 罗泽 南凯 阎保山 基于发布/订阅系统的事件服务体系结构[J] 计算机工程与应用 2003.27
8. WATSON P. Database and the Grid[R]. Technical Report CS-TR-755. University of Newcastale,2001
9. Java Programmer’s Guide Core Framework, http://www-unix.globus.org/toolkit/3.0/ogsa/docs/java_programmers_guide.html
10. 柴 晓 路 EAI 和 Web 服 务 - 轻 松 进 行 企 业 应 用 集 成 [EB/OL]. http://www-900.ibm.com/developerWorks/cn/webservices/ws-eai/cxl/
11. 任捷,吴明晖,应晶 Web Services 技术在异构系统集成中的应用研究[J]。计算机工程与应用 2004,(1).
12. Neil Hardman, Andrew Borley, James Magowan. OGSA-DAI: A look under the hood: Part 1: Architecture and database access[EB/OL]. http://www-106.ibm.com/developerworks/grid/library/gr-ogsadai. 2004-03-16.
13 Neil Hardman, Andrew Borley, James Magowan. OGSA-DAI: A look under the hood: Part 2: Activities and results [EB/OL]. http://www-106.ibm.com/developerworks/grid/library/gr-ogsadai2.2004-0316.
14. OGSA-DAI Team. Installation Guide for Release 3.1 of the OGSA-DAI Software[M/CD]. OGSA-DAI-USER-INSTALL-v3.15
15. M.Nedim Alpdemir, Arijit Mukherjee, Anstasios Gounaris. OGSA-DQP:A Service for Distributed Querying on the Grid[EB/OL]. http://www.cs.man.ac.uk/~gounaris/EDBTDemopaper.pdf
16. 任捷,吴明晖. 应晶 Web Services 技术在异构系统集成中的应用研究[J].计算机工程与应用 2004,(1).
17. The Globus Toolkit 3 Programmer's Tutorial. http://www.casa-sotomayor.net/gt3-tutorial-working/
18. 李彬. 用 GT3 开发网格服务. http://www-900.ibm.com/developerWorks/cn/webservices/ws-gt3/index.shtml
19. 石骁騑. Oracle 8 高级数据复制技术 http://www0.ccidnet.com/tech/guide/2001/03/28/58_1900.html
20. 闪四清. SQL Server 数据库技术 http://edu.chinaz.com/2004/8-2/4440.asp
21. Sybase 系统的数据同步与复制. http://www.xiaotong-db.com.cn/sjfz/sybase/SYBASE_tb_fz.html
22. 史隆,都志辉. 网格数据库管理模型与策略. 计算机科学,2004(31):12-15
23. 汪锦岭,金莅弘. 网格环境下的数据库系统. 计算机科学,2004(31):4-6
24. 何杨平,韩海雯,沈波. 一种基于数据库复制技术的分布式联机方案. 微机发展,2004(7):33-35
25. 罗泽,南凯,阎保山. 数据网格事件服务中基于内容的消息路由. 微电子学与计算机,2003(12):27-31
26. 史海波,马玉林. 分布数据复制技术及其应用. 计算机工程与应用. 2002(2):171-173
27. 王婉菲,张志浩. 分布式数据库系统的复制机制及应用. 计算机工程与科学. 2003(25):88-91
28. Milenko Petrovic,Ioana Burcea,Hans-Arno Jacbsen. S-ToPSS:Semantic Toronto Publish/Subscribe System. http://www.eecg.toronto.edu/~jacobsen
29. Arno Jocobsen. Publish/Subscribe:Applications,Concepts,and Algorithms. http://www.eecg.toronto.edu/~jacobsen
30. L.M.Haas,R.J.Miller,Niswonger.Transforming Heterogeneous Data With Database Middleware:Beyond Integration.Bulletin of the IEEE Computer Society Technical Committe on Data Engineering
31. 都志辉,陈渝,刘鹏. 网格计算.第一版. 出版地:清华大学出版社,2002
32. Grid Services. http://www.ogsadai.org.uk/docs/R4.0/doc/GridServices.html
33. OGSA-DAI Overview. http://www.ogsadai.org.uk/docs/R4.0/doc/DAIOverview.html
34. DAI Service Group Registry. http://www.ogsadai.org.uk/docs/R4.0/doc/DAISGR.html
35. Grid Data Service Factory. http://www.ogsadai.org.uk/docs/R4.0/doc/GDSF.html
36. Grid Data Service. http://www.ogsadai.org.uk/docs/R4.0/doc/GDS.html
37. Susanne Busse,Ralf-Detlef Kutsche,Ulf Leser. Federated Information Systems:Concepts,Terminology and Architectures. http://citeseer.ist.psu.edu/busse99federated.html
38. Vijayshankar Raman,Inderpal Narang,Chris Crone. Data Access and Management Services on Grid.
39. Steven Tuecke,Karl Czajkowski,Ian Foster. Grid Service Specification. http://www.gridforum.org/ogsi-wg
40. Ian Foster,Carl Kesselman,jeffrey M.Nick. The Physiology of the Grid. http://www.globus.org/research/papers/ogsa.pdf
41. Yannis Papakonstantinou,Hector Garcia-Molina,Jennifer Widom. Object Exchange Across Heterogeneous Information Sources.
42. Ian Foster,Carl Kesselman,Steven Turche.The Anatomy of the Grid. http://www.globus.org/research/papers/anatomy.pdf
43. Hector Carcia-Molina,Yannis Papakonstantinou,Dallan Quass.The TSIMMIS Approach to Mediation:Data Models and Languages. http://citeseer.ist.psu.edu/12944.html
44. M.Nedim Alpdemir,Arijit Mukherjee,Norman W.Paton.Servic-Based Distributed Querying on the Grid.
45. 高卫中,廖华明. 基于网格服务的带分支的实时监控复制技术及实现. 计算机工程与应用,2005(51):104-107
46. 高卫中,廖华明. 网格数据库中基于发布/订阅的分片复制技术及实现. 计算机应用,2005(6):
47. 高卫中,廖华明. 数据网格中基于发布/订阅的实时复制技术及实现. 微型机与应用,2005(6):
48. 肖侬,付伟,黄斌,卢锡城. Griddaen 数据网格系统的设计与关键技术实现.http://www.chinagrid.net/grid/paperppt/Griddaen.doc
49. Laura Haas,Eileen Lin. IBM Federated Database Technology. http://www-900.ibm.com/developerWorks/cn/dmdd/library/techarticles/0203haas/0203haas_eng.shtml
50. GGF 的 OGSI 工作组, http://www.gridforum.org/ogsi-wg/
51. OGSA 规范, http://www.gridforum.org/ogsi-wg/drafts/GS_Spec_draf03_2002-07-17.pdf
52. OGSA 结构描述, http://www.gridforum.org/ogsi-wg/drafts/ogsa_draft2.9_2002-06-22.pdf,http://www.Globus.org/ogsa
53. 网格计算究竟离我们有多远, http://www.zdnet.com.cn/techupdate/network/intranet/story/0,3800030561,39212615,00.htm
54. 专家访谈:Carol Rigdon,IBM 高级技术支持, http://www-900.ibm.com/developerWorks/cn/dmdd/library/techarticles/0306rigdon/0306rigdon.shtml