达梦数据库系统动态数据复制技术研究
|
摘要
数据复制是分布式数据库中提高系统可用性和可靠性的一项重要技术。长期以来,数据对象均使用静态复制方式,数据复制计划由分布式数据库管理者指定,该计划是固定的,直到管理者执行手动重新指定。然而,如果数据对象的读-写模式发生动态的、不可预期的变化时,静态的复制计划将导致严重的性能问题。理想的复制计划要能够根据读-写模式的变化自动地进行适当调整。因此,动态数据复制技术是分布式数据库系统面临的一个新的课题。
在主复制方法的基础上,引入时间段限制,扩展成了一种具有自适应性的动态数据复制方法。采用“读一个写所有”更新策略,并假设在大多数情况,下一段时间内的读-写模式,能根据其紧跟着的上一段时间内的读-写模式进行预测。采用这种方法,可以通过比较一个时间段后,数据对象的本地读次数和远程写次数的大小,来自动调整该数据对象的复制计划。
此外,借鉴了Microsoft SQL Server的复制模型,设计了由发布者、订阅者、发布、订阅、项目和数据源共同组成的复制模型;同时引入了“拉”式(Pull)订阅模型结构,实现了由订阅者启动复制过程完成数据库到数据库的复制;采用触发器技术,监视发布者的数据更改,解决了数据更改的捕捉问题;采用流行的可扩展标识语言(eXtensible Markup Language,XML)技术(包括创建与解析)很好地解决了复制数据更改的网络传播问题。
Data replication is an important technology which is often used to improve the usability and dependability of distributed database system. Traditionally, Data objects are replicated in static way, their replication schemes are assigned by database administrator; the schemes are static and unaltered during the replication process unless they are changed by administrator. However, if the read-write patterns change dynamically, in unpredictable ways, a static replication scheme may lead to severe performance problem. So, dynamic data replication is a new subject of distributed database system.
In this paper, we present a distributed algorithm for dynamic data replication of an object in a distributed database system. The algorithm dynamically adapts the replication scheme of an object to the pattern of read-write requests in the distributed database system. It works in the read-one-write-all context, which means a file read involves only one replica and the write update is propagated to all the replica; the read-write pattern during a time period is in most cases predictable based on the read-write pattern in the immediately preceding time periods.
Referred to replication model of Microsoft SQL Server, our replication model has been developed as six parts including publisher, subscriber, publication, subscription, data item and data source. The introduction of PULL subscribing structure fulfills bidirectional replication between database and database which is started by subscriber.
In addition, the usage of trigger can monitor the data changes in publisher’s database, thus the problem of catching the data changes has been worked out. And with the popular XML techniques (creating and parsing), transmitting replication data among networks has been better resolved.
在主复制方法的基础上,引入时间段限制,扩展成了一种具有自适应性的动态数据复制方法。采用“读一个写所有”更新策略,并假设在大多数情况,下一段时间内的读-写模式,能根据其紧跟着的上一段时间内的读-写模式进行预测。采用这种方法,可以通过比较一个时间段后,数据对象的本地读次数和远程写次数的大小,来自动调整该数据对象的复制计划。
此外,借鉴了Microsoft SQL Server的复制模型,设计了由发布者、订阅者、发布、订阅、项目和数据源共同组成的复制模型;同时引入了“拉”式(Pull)订阅模型结构,实现了由订阅者启动复制过程完成数据库到数据库的复制;采用触发器技术,监视发布者的数据更改,解决了数据更改的捕捉问题;采用流行的可扩展标识语言(eXtensible Markup Language,XML)技术(包括创建与解析)很好地解决了复制数据更改的网络传播问题。
Data replication is an important technology which is often used to improve the usability and dependability of distributed database system. Traditionally, Data objects are replicated in static way, their replication schemes are assigned by database administrator; the schemes are static and unaltered during the replication process unless they are changed by administrator. However, if the read-write patterns change dynamically, in unpredictable ways, a static replication scheme may lead to severe performance problem. So, dynamic data replication is a new subject of distributed database system.
In this paper, we present a distributed algorithm for dynamic data replication of an object in a distributed database system. The algorithm dynamically adapts the replication scheme of an object to the pattern of read-write requests in the distributed database system. It works in the read-one-write-all context, which means a file read involves only one replica and the write update is propagated to all the replica; the read-write pattern during a time period is in most cases predictable based on the read-write pattern in the immediately preceding time periods.
Referred to replication model of Microsoft SQL Server, our replication model has been developed as six parts including publisher, subscriber, publication, subscription, data item and data source. The introduction of PULL subscribing structure fulfills bidirectional replication between database and database which is started by subscriber.
In addition, the usage of trigger can monitor the data changes in publisher’s database, thus the problem of catching the data changes has been worked out. And with the popular XML techniques (creating and parsing), transmitting replication data among networks has been better resolved.
引文
[1]林伟,张延园,董欢庆.企业级数据复制平台的构建方案.计算机应用研究,2005,10(3):88～90
[2]者敬.开放式异构数据库复制框架的研究:[博士学位论文]。北京:中国科学院软件研究所,2002.
[3]周英飚,卫国.一致性数据复制.计算机工程与应用,2004, 40(36):183～185
[4] Yixiu Huang, Ouri Wolfson. A Competitive Dynamic Data Replication Algorithm. in: IEEE Proc. 9th International Conference on Data Engineering: 1993. 310～317
[5] John Kubiatowicz, David Bindel, Yan Chen et al. OceanStore: An Architecture for Global-Scale Persistent Storage. in: ACM Press ed. ACM SIGARCH Computer Architecture News. New York, NY, USA: ACM Press,2000.190～201
[6] Zhihong Lu, Kathryn S. McKinley. Partial Collection Replication versus Caching for Information Retrieval Systems. in: Emmanuel, Yannakoudakis eds. Annual ACM Conference on Research and Development in Information Retrieval. New York, NY, USA: ACM Press, 2000.248～255
[7] BRIAN F. COOPER, HECTOR GARCIA-MOLINA. Peer-to-Peer Data Trading to Preserve Information. ACM Transactions on Information Systems, April 2002, 20(2): 133～170
[8] Qin Lv, Pei Cao, Edith Cohen et al. Search and Replication in Unstructured Peer-to-Peer Networks. in: Kemal, Ebcioglu eds. International Conference on Supercomputing. New York, NY, USA: ACM Press, 2002.84～95
[9]燕彩蓉,谢怡.集群Web服务器的几种数据复制解决方案.计算机应用研究,2003,20(5):119～122
[10]焦文英,邓苏.面向资源广域分布环境的数据复制定位.计算机工程,2003,29(14):81～83
[11] Brian Cooper, Hector Garcia-Molina. Creating trading network of digital archives. in: Edward A. Fox, Christine L. Borgman eds. International Conference on Digital Libraries. New York, NY, USA: ACM Press, 2001. 353～362
[12] Takahiro Hara, Yin-Huei Loh, Shojiro Nishio. Data replication methods based on the stability of radio links in Ad Hoc networks. in: IEEE Computer Society eds. Proceedings of the 14th International Workshop on Database and Expert Systems Applications (DEXA’03). Washington, DC, USA: IEEE Computer Society, 2003. 969～973
[13] Rene Dufrene. Data replication over the WAN. Computer Technology Review. Oct 2003, 23(10): 35～36
[14] Cristiana Amza, Alan L. Cox, Willy Zwaenepoel. Data replication strategies for fault tolerance and availability on commodity clusters. in: IEEE Computer Society ed. Proceedings of the 2000 International Conference on Dependable Systems and Networks (formerly FTCS-30 and DCCA-8). Washington, DC, USA: IEEE Computer Society, 2000.459～467
[15]王明钟,李战怀,董欢庆.基于层次结构的远程数据复制系统.微电子学与计算机,2005,22(5):98～101
[16] Houda Lamehamedi, Boleslaw Szymanski, Zujun Shentu et al. Data Replication Strategies in Grid Environments. in: IEEE Computer Society ed. Proceedings of the Fifth International Conference on Algorithms and Architectures for Parallel Processing (ICA3PP.02). Washington, DC, USA: IEEE Computer Society, 2002. 378～385
[17] S. Palazzo, A. Puliafito, M. Scarpa. Design and evaluation of a replicated database of mobile systems. Wireless Networks, 2000, 36(6): 131～144
[18] Rosalia Christodoulopoulou, Reza Azimi, Angelos Bilas. Dynamic data replication: an approach to providing fault-tolerant shared memory clusters. in: IEEE Computer Society ed. Proceedings of the Ninth International Symposium on High-PerformanceComputer Architecture (HPCA-9’03). Washington, DC, USA: IEEE Computer Society, 2003. 203～214
[19] Jianliang Xu, Bo Li, Dik Lun Lee. Placement Problems for Transparent Data Replication Proxy Services. IEEE Journal On Selected Areas In Communications, September 2002, 20(7): 1383～1398
[20] David Ratner, Peter Reiher, Gerald J. Popek et al. Replication Requirements in Mobile Environments. Mobile Networks and Applications, 2001, 33(6): 525～533
[21]汪千松,李骥.基于Oracle9i的复制技术及应用.计算机工程与设计,2005,26(7):1971～1973
[22] Microsoft Corporation.复制与自然语言查询.第1版.李宏益,杨丹,景京.北京:清华大学出版社,2001. 3～64
[23]黄求根,吴宛萍.分布式环境下基于事务的数据复制及其应用.计算机应用研究,2000,17(12):100～103
[24]史海波,马玉林.分布数据复制技术及其应用.计算机工程与应用,2002,38(2):171～173
[25]盖九宇,张忠能,肖鹤.分布式数据库数据复制技术的分析与应用.计算机应用与软件,2005,22(7):36～38
[26]王晓峰,李宛洲.广域网系统中的数据复制方案.计算机工程,2001,27(4):116～117
[27]钱文海,贺元启.基于数据复制技术的移动数据库系统.计算机应用研究,2002,19(7):50～53
[28] T Loukopoulos, I Ahmad. Static and Adaptive Data Replication Algorithms for Fast Information Access in Large Distributed Systems. in: The 20th International Conference on Distributed Computing Systems. Taipei: 2000. 385～392
[29]从丽晖,夏秀峰,于戈.一种基于分布式环境的副本自动审视技术研究.计算机工程与应用,2003,39(4):192～194
[30] Ouri Wolfson, Sushil Jajodia, Yixiu Huang. An Adaptive Data Replication Algorithm. ACM Transactions on Database Systems, 1997, 22(2): 255～314
[31] Giacomo Cabri, Antonio Corradi, Franco Zambonelli. Experience of Adaptive Replication in Distributed File Systems. in: Proc. of EUROMICRO-22: 1996. 459～466
[32] S. Acharya, S. B. Zdonik. An Efficient Schema for Dynamic Data Replication. Brown University CS, 1993: 43～93
[33] Y. Bartal, A. Fiat, Y. Rabani. Competitive algorithms for distributed data management. in: Proc. 24th Annual ACM Symp. On Theory of Computing. New York, NY: ACM Press, 1992. 39～50
[34]杜杏菁.分布式数据库中复制技术的研究.华北科技学院学报,2004,1(3):88～91
[35]吴永英,李逸.分布式数据库管理系统复制技术的设计与实现. [硕士学位论文]。武汉:华中科技大学图书馆,2001.
[36] J. Holiday, D. Agrawal, A. EI. Abbadi. Partial database replication using epidemic communication. In: Luis E.T.Rodrigues, Universidate de Lisboa, Portugal. eds. Proceedings of the 22nd international Conference on Distributed Computing Systems. Los Alamitos, California: IEEE Computer Society, 2002. 485～493
[37]朱光宇,李松年,刘坤等.一个基于XML的数据收集与分发方法.计算机工程,2002,28(1):257～259
[38]乐鑫喜,刘清.异构数据库集成系统的设计与实现.计算机工程与应用,2005,41(27):174～177
[39]陈传波,李波.基于XML的数据复制技术的研究. [硕士学位论文]。武汉:华中科技大学图书馆,2004.
[40]王欣,左春.企业级数据复制平台的构建方案.计算机工程与应用,2003,39(3):198～200
[2]者敬.开放式异构数据库复制框架的研究:[博士学位论文]。北京:中国科学院软件研究所,2002.
[3]周英飚,卫国.一致性数据复制.计算机工程与应用,2004, 40(36):183～185
[4] Yixiu Huang, Ouri Wolfson. A Competitive Dynamic Data Replication Algorithm. in: IEEE Proc. 9th International Conference on Data Engineering: 1993. 310～317
[5] John Kubiatowicz, David Bindel, Yan Chen et al. OceanStore: An Architecture for Global-Scale Persistent Storage. in: ACM Press ed. ACM SIGARCH Computer Architecture News. New York, NY, USA: ACM Press,2000.190～201
[6] Zhihong Lu, Kathryn S. McKinley. Partial Collection Replication versus Caching for Information Retrieval Systems. in: Emmanuel, Yannakoudakis eds. Annual ACM Conference on Research and Development in Information Retrieval. New York, NY, USA: ACM Press, 2000.248～255
[7] BRIAN F. COOPER, HECTOR GARCIA-MOLINA. Peer-to-Peer Data Trading to Preserve Information. ACM Transactions on Information Systems, April 2002, 20(2): 133～170
[8] Qin Lv, Pei Cao, Edith Cohen et al. Search and Replication in Unstructured Peer-to-Peer Networks. in: Kemal, Ebcioglu eds. International Conference on Supercomputing. New York, NY, USA: ACM Press, 2002.84～95
[9]燕彩蓉,谢怡.集群Web服务器的几种数据复制解决方案.计算机应用研究,2003,20(5):119～122
[10]焦文英,邓苏.面向资源广域分布环境的数据复制定位.计算机工程,2003,29(14):81～83
[11] Brian Cooper, Hector Garcia-Molina. Creating trading network of digital archives. in: Edward A. Fox, Christine L. Borgman eds. International Conference on Digital Libraries. New York, NY, USA: ACM Press, 2001. 353～362
[12] Takahiro Hara, Yin-Huei Loh, Shojiro Nishio. Data replication methods based on the stability of radio links in Ad Hoc networks. in: IEEE Computer Society eds. Proceedings of the 14th International Workshop on Database and Expert Systems Applications (DEXA’03). Washington, DC, USA: IEEE Computer Society, 2003. 969～973
[13] Rene Dufrene. Data replication over the WAN. Computer Technology Review. Oct 2003, 23(10): 35～36
[14] Cristiana Amza, Alan L. Cox, Willy Zwaenepoel. Data replication strategies for fault tolerance and availability on commodity clusters. in: IEEE Computer Society ed. Proceedings of the 2000 International Conference on Dependable Systems and Networks (formerly FTCS-30 and DCCA-8). Washington, DC, USA: IEEE Computer Society, 2000.459～467
[15]王明钟,李战怀,董欢庆.基于层次结构的远程数据复制系统.微电子学与计算机,2005,22(5):98～101
[16] Houda Lamehamedi, Boleslaw Szymanski, Zujun Shentu et al. Data Replication Strategies in Grid Environments. in: IEEE Computer Society ed. Proceedings of the Fifth International Conference on Algorithms and Architectures for Parallel Processing (ICA3PP.02). Washington, DC, USA: IEEE Computer Society, 2002. 378～385
[17] S. Palazzo, A. Puliafito, M. Scarpa. Design and evaluation of a replicated database of mobile systems. Wireless Networks, 2000, 36(6): 131～144
[18] Rosalia Christodoulopoulou, Reza Azimi, Angelos Bilas. Dynamic data replication: an approach to providing fault-tolerant shared memory clusters. in: IEEE Computer Society ed. Proceedings of the Ninth International Symposium on High-PerformanceComputer Architecture (HPCA-9’03). Washington, DC, USA: IEEE Computer Society, 2003. 203～214
[19] Jianliang Xu, Bo Li, Dik Lun Lee. Placement Problems for Transparent Data Replication Proxy Services. IEEE Journal On Selected Areas In Communications, September 2002, 20(7): 1383～1398
[20] David Ratner, Peter Reiher, Gerald J. Popek et al. Replication Requirements in Mobile Environments. Mobile Networks and Applications, 2001, 33(6): 525～533
[21]汪千松,李骥.基于Oracle9i的复制技术及应用.计算机工程与设计,2005,26(7):1971～1973
[22] Microsoft Corporation.复制与自然语言查询.第1版.李宏益,杨丹,景京.北京:清华大学出版社,2001. 3～64
[23]黄求根,吴宛萍.分布式环境下基于事务的数据复制及其应用.计算机应用研究,2000,17(12):100～103
[24]史海波,马玉林.分布数据复制技术及其应用.计算机工程与应用,2002,38(2):171～173
[25]盖九宇,张忠能,肖鹤.分布式数据库数据复制技术的分析与应用.计算机应用与软件,2005,22(7):36～38
[26]王晓峰,李宛洲.广域网系统中的数据复制方案.计算机工程,2001,27(4):116～117
[27]钱文海,贺元启.基于数据复制技术的移动数据库系统.计算机应用研究,2002,19(7):50～53
[28] T Loukopoulos, I Ahmad. Static and Adaptive Data Replication Algorithms for Fast Information Access in Large Distributed Systems. in: The 20th International Conference on Distributed Computing Systems. Taipei: 2000. 385～392
[29]从丽晖,夏秀峰,于戈.一种基于分布式环境的副本自动审视技术研究.计算机工程与应用,2003,39(4):192～194
[30] Ouri Wolfson, Sushil Jajodia, Yixiu Huang. An Adaptive Data Replication Algorithm. ACM Transactions on Database Systems, 1997, 22(2): 255～314
[31] Giacomo Cabri, Antonio Corradi, Franco Zambonelli. Experience of Adaptive Replication in Distributed File Systems. in: Proc. of EUROMICRO-22: 1996. 459～466
[32] S. Acharya, S. B. Zdonik. An Efficient Schema for Dynamic Data Replication. Brown University CS, 1993: 43～93
[33] Y. Bartal, A. Fiat, Y. Rabani. Competitive algorithms for distributed data management. in: Proc. 24th Annual ACM Symp. On Theory of Computing. New York, NY: ACM Press, 1992. 39～50
[34]杜杏菁.分布式数据库中复制技术的研究.华北科技学院学报,2004,1(3):88～91
[35]吴永英,李逸.分布式数据库管理系统复制技术的设计与实现. [硕士学位论文]。武汉:华中科技大学图书馆,2001.
[36] J. Holiday, D. Agrawal, A. EI. Abbadi. Partial database replication using epidemic communication. In: Luis E.T.Rodrigues, Universidate de Lisboa, Portugal. eds. Proceedings of the 22nd international Conference on Distributed Computing Systems. Los Alamitos, California: IEEE Computer Society, 2002. 485～493
[37]朱光宇,李松年,刘坤等.一个基于XML的数据收集与分发方法.计算机工程,2002,28(1):257～259
[38]乐鑫喜,刘清.异构数据库集成系统的设计与实现.计算机工程与应用,2005,41(27):174～177
[39]陈传波,李波.基于XML的数据复制技术的研究. [硕士学位论文]。武汉:华中科技大学图书馆,2004.
[40]王欣,左春.企业级数据复制平台的构建方案.计算机工程与应用,2003,39(3):198～200