达梦数据库系统备份与恢复机制研究
|
摘要
数据库管理系统会因一些不可预知的软硬件故障影响事务的正确执行甚至破坏数据库,降低了系统的可靠性和可用性。数据库系统的备份与恢复部分正是要解决这些问题。本课题以“大型通用数据库管理系统DM V5.6研发及应用”为背景,系统深入地研究了数据库的备份与恢复技术,并改进了达梦备份与恢复子系统,以提高达梦数据库的可靠性和可用性。
在现有达梦数据库备份和恢复子系统的基础上设计并实现了备份和日志多文件、异步检查点、基于时间点的恢复和并行恢复。备份多文件是一次备份可根据需要产生多个备份文件,使得备份文件的转移变得容易。日志多文件是指日志系统采用多个日志文件循环使用,避免了系统单个日志文件不断增长的问题。
异步检查点采取对脏页上S锁,脏页刷盘完毕立即释放锁的策略,来实现对数据缓冲区的刷盘,使得检查点期间,事务能照常更新。
基于时间点恢复采用在系统中添加时间类型日志,并在物理事务结束时,写入时间类型日志来标识物理事务结束时间的方法,只重做指定时间点以内的物理事务,使数据库恢复到时间点的一致性状态。
并行恢复采取基于页号取余的日志记录分配算法,一个日志读取线程多个日志重做线程的框架,使得对一个数据页的日志重做由单个线程完成,保证了数据库的一致性。多个日志重做线程并行执行,充分利用了系统的I/O能力,缩短了系统恢复的时间。
实验表明异步检查点能提高系统的吞吐量,缩短响应时间。并行恢复能利用系统的I/O并行能力,缩短恢复的时间。
The unpredictable software and hardware failures can influence the implementation of transactions or even undermine the database in database management system, and reduce its reliability and availability. Backup and recovery subsystem is to solve these problems. Subject to the“large-scale development and application of generic database management system DM V5.6”as background, we study the database backup and recovery technology and improve backup and recovery subsystem to improve the reliability and availability.
Based on the existing DM database backup and recovery subsystem we work on the four main areas, including backup and log multi-file, asynchronous checkpoint, time-based recovery and parallel recovery. Backup multi-file is that a backup operation can produce a number of backup files if need, making the transfer of backup files easier. Log multi-file uses two or more log files circularly to avoid the problem of ever-increasing individual log file.
When setting asynchronous checkpoint, we S lock a dirty page, flush the dirty page, then release the S lock after the flushing finished. The transactions can also update pages in the interval of asynchronous checkpoint.
Time-based recovery creates a time type log and inserts a time type log in the end of a physical transaction to mark the end time of the physical transaction. When using time-based recovery, we only need to redo the physical database whose time is less than the time point, and back to consistency condition at the time point.
Parallel recovery is based on page number mode dispatching algorithm and using the framework of one log-reading thread and multi redo thread, ensuring the log records on one page redoed by a thread, multi redo threads run parallelly. It can make full use of the I/O capability of the system to shorten recovery time.
Experiments show asynchronous checkpoint can improve the throughput and shorten response time. Parallel recovery can use the ability of system I/O to shorten recovery time.
在现有达梦数据库备份和恢复子系统的基础上设计并实现了备份和日志多文件、异步检查点、基于时间点的恢复和并行恢复。备份多文件是一次备份可根据需要产生多个备份文件,使得备份文件的转移变得容易。日志多文件是指日志系统采用多个日志文件循环使用,避免了系统单个日志文件不断增长的问题。
异步检查点采取对脏页上S锁,脏页刷盘完毕立即释放锁的策略,来实现对数据缓冲区的刷盘,使得检查点期间,事务能照常更新。
基于时间点恢复采用在系统中添加时间类型日志,并在物理事务结束时,写入时间类型日志来标识物理事务结束时间的方法,只重做指定时间点以内的物理事务,使数据库恢复到时间点的一致性状态。
并行恢复采取基于页号取余的日志记录分配算法,一个日志读取线程多个日志重做线程的框架,使得对一个数据页的日志重做由单个线程完成,保证了数据库的一致性。多个日志重做线程并行执行,充分利用了系统的I/O能力,缩短了系统恢复的时间。
实验表明异步检查点能提高系统的吞吐量,缩短响应时间。并行恢复能利用系统的I/O并行能力,缩短恢复的时间。
The unpredictable software and hardware failures can influence the implementation of transactions or even undermine the database in database management system, and reduce its reliability and availability. Backup and recovery subsystem is to solve these problems. Subject to the“large-scale development and application of generic database management system DM V5.6”as background, we study the database backup and recovery technology and improve backup and recovery subsystem to improve the reliability and availability.
Based on the existing DM database backup and recovery subsystem we work on the four main areas, including backup and log multi-file, asynchronous checkpoint, time-based recovery and parallel recovery. Backup multi-file is that a backup operation can produce a number of backup files if need, making the transfer of backup files easier. Log multi-file uses two or more log files circularly to avoid the problem of ever-increasing individual log file.
When setting asynchronous checkpoint, we S lock a dirty page, flush the dirty page, then release the S lock after the flushing finished. The transactions can also update pages in the interval of asynchronous checkpoint.
Time-based recovery creates a time type log and inserts a time type log in the end of a physical transaction to mark the end time of the physical transaction. When using time-based recovery, we only need to redo the physical database whose time is less than the time point, and back to consistency condition at the time point.
Parallel recovery is based on page number mode dispatching algorithm and using the framework of one log-reading thread and multi redo thread, ensuring the log records on one page redoed by a thread, multi redo threads run parallelly. It can make full use of the I/O capability of the system to shorten recovery time.
Experiments show asynchronous checkpoint can improve the throughput and shorten response time. Parallel recovery can use the ability of system I/O to shorten recovery time.
引文
[1] W. Wihl. The Impact of Recovery on Concurrency Control. in: Avi Silberschatz eds.. Proceedings of the Eighth ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems. Philadelphia, Pennsylvania, United States. 1989. 1515 Broadway, 17th Floor New York, NY USA: ACM Press, 1989. 259~269
[2] R. Agrawal, D. DeWitt. Integrated Concurrency Control and Recovery Mechanisms: Design and Performance Evaluation. ACM Transactions on Database Systems, 1985, 10(4): 529~564
[3] Andreas Reuter. Performance Analysis of Recovery Techniques. ACM Transactions on Database Systems, 1984, 9(4): 526~559
[4] J. S. Keen, W. J. Dally. Performance Evaluation of Ephemeral Logging. in: Peter Buneman, Sushil Jajodia eds.. Preceedings of the 1993 ACM SIGMOD International Conference on Management of Data. Washington, D.C., United States. 1993. 1515 Broadway, 17th Floor New York, NY USA: ACM Press, 1993. 187~196
[5] Richard Dean Regan. High Performance Ephermeral Logging for Database Systems: [Dissertation for the Degree of Doctor of Philosophy (Computer Science)]. Michigan 48106-1346 USA: Polytechnic University, 2001
[6] J. S. Keen, W. J. Dally. XEL: Extended Ephemeral Logging for Log Storage Managerment. in:N. R. Adam, B. K. Bhargava, Yelena Yesha eds.. Proceedings of the Third International Conference on Information and Knowledge Management. Gaithersburg, Maryland, United States. 1994. 1515 Broadway, 17th Floor New York, NY USA: ACM Press, 1994. 312~321
[7] J. S. Keen, W. J. Dally. Extended Ephemeral Logging: Log Storage Management for Application with Long Lived Transactions. ACM Transactions on Database Systems, 1997, 22(1): 1~42
[8]宋广华,杨长生.基于混合日志的内存数据库恢复子系统.浙江大学学报(理学报), 2001, 28(2): 164~168
[9] Taesoon Park, Junguk L. Kim. A Checkpointing Recovery Approach in A Distributed System on the CSMA/CD Network. in: Hal Berghel, Ed Deaton, George Hedrick, David Roach, Roger Wainwright eds.. Proceedings of the 1992 ACM/SIGAPP Symposium on Applied Computing. Kansas City, Missouri, United States. 1992. 1515 Broadway, 17th Floor New York, NY USA: ACM Press, 1992. 614~623
[10] Raj Sekhar Pamnla, P. K. Srimeni. Check Pointing Strategies for Database Systems. in: Pat Davis, Vicki McClintock eds.. Proceedings of the 15th Annual Conference on Computer Science. St. Louis, Missouri, United States, 1987. 1515 Broadway, 17th Floor New York, NY USA: ACM Press, 1987. 88~97
[11] Theo Haerder, Andreas Reuter. Principles of Transaction Oriented Database Recovery. Computing Surveys, 1983, 15(4): 287~317
[12]于秀云.对数据库恢复策略的性能分析.计算机研究与发展, 1998, 35(12): 1135~1141
[13] J. Gray, P. McJones, M. Blasgen, et al. The Recovery Manager of the System R Database Manager. ACM Computing Surveys, 1981, 13(2): 223~243
[14] Hector Garcia Molina, J. D. Ullman, Jennifer Widom.数据库系统实现.第一版.杨冬青,唐世渭,徐其钧.北京:机械工业出版社, 2002. 304~331
[15] A. Reuter. A Fast Transaction-Oriented Logging Scheme for Undo Recovery. IEEE Transactions on Software Engineering, 1980, 6(4): 348~356
[16] David Lomet, Mark Tuttle. A Theory of Redo Recovery. in: Yannis Papakonstantinou, Alon Halevy, Zachary Ives eds.. Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data. San Diego, California. 2003. 1515 Broadway, 17th Floor New York, NY USA: ACM Press, 2003. 397~406
[17] C. Mohan, D. Haderle, B. Lindsay, et al. ARIES: A Transaction Recovery MethodSupporting Fine-Granularity Locking and Partial Rollbacks Using Write-Ahead Logging. ACM Transactions on Database Systems, 1992, 17(1): 94~162
[18]施晓秋. ARIES数据库恢复算法的分析.温州大学学报, 2000, 37(2): 46~50
[19]陈宇,孟煊,王燕. DB2数据库恢复策略及实现.西南民族学院学报(自然科学版), 1999, 25(1): 26~29
[20] Rama Velpufi, Anand Adkoli. Oracle8i备份与恢复手册.第一版.蒋蕊,王磊,王毳等.北京:机械工业出版社, 2001. 117~252
[21] Dean Kuo. Model and Verification of A Data Manager Based on ARIES. ACM Transactions on Database Systems (TODS), 1996, 21(4): 427~479
[22]王意洁,胡守仁.一种实用有效的恢复策略WAL_P.国防科技大学学报, 2000, 22(1): 7~10
[23] C. Mohan, I. Narang. ARIES/CSA: A Method for Database Recovery in Client-Server Architechtures. ACM SIGMOD Record, 1994, 23(2): 55~66
[24]黄志华. Oracle的故障恢复机制.漳州师范学院学报(自然科学版), 2002, 15(3): 29~32
[25]冯玉才.数据库系统基础.第二版.武汉:华中理工大学出版社, 1993. 29~30
[26]曾利平,刘新华. Oracle9i数据库的备份与恢复.计算机与现代化, 2002, 3(12): 62~64
[27]郭鲜凤,张和义. SQL Server分部式数据库的并发控制和故障恢复.科技情报开发与经济, 2002, 12(1): 60~61
[28] Raghu Ramakrishnan, Johannes Gehrke.数据库管理系统.第二版.周立柱,蒋旭东,石晶等.北京:清华大学出版社, 2002. 431~445
[29]阳富民,冯玉才,吴恒山等.一种分部式DBMS故障恢复技术.计算机研究与发展, 1995, 32(5): 25~30
[30]许蔚.分部式数据库DM3并发控制和故障恢复研究:[硕士学位论文]。武汉:华中理工大学图书馆,2000
[31] Vassos Hadzilacos. A Theory of Reliability in Database Systems. Journal of the Association for Computing Machinery, 1988, 35(1): 121~145
[32] Michael Stone Braker. Readings in Database Systems. First Edition. 340 Pine Street Sixth Floor San Francisco, CA USA: Morgan Kaufmann Publisher Inc, 1988. 140~150
[33]萨师渲,王珊.数据库系统概论.第三版.北京:高等教育出版社, 2000. 247~262
[34] Heiko Schuldt, Gustavo Alonso, Hans-Jorg Schek. Concurrency Control and Recovery in Transactional Process Management. in: Victor Vianu, Christors Papadimitriou eds.. Proceedings of the Eighteenth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems. Philadelphia, Pennsylvania, United States. 1999. 1515 Broadway, 17th Floor New York, NY USA: ACM Press, 1999. 316~326
[35] Abraham Silberschatz, H. F. Korth, S. Sudarshan.数据库系统概念.第三版.杨冬青,唐世渭.北京:机械工业出版社, 1999. 359~379
[36] D. B. Lomet. High Speed On-line Backup when Using Logical Log Operations. in: Maggie Dunham, J. F. Naughtom, Weidong Chen, Nick Koudas eds.. International Conference on Management of Data and Symposium on Principles of Database Systems. Dallas, Texas, United States. 2000. 1515 Broadway, 17th Floor New York, NY USA: ACM Press, 2000. 34~45
[37] David Lomet, Mark Tuttle. Logical Logging to Extend Recovery to New Domains. in: Susan B. Davidson, Christos Faloutsos eds.. Proceedings of the 1999 ACM SIGMOD International Conference on Management of Data. Philadelphia, Pennsylvania, United States. 1999. 1515 Broadway, 17th Floor New York, NY USA: ACM Press, 1999. 73~84
[38] G. Alonso, D. Agrawal, A. E. Abbadi. Reducing Recovery Constraints on Locking Based Protocols. in: Victor Vianu ed. Proceedings of the Thirteenth ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems. Minneapolis, Minnesota, United States. 1994. 1515 Broadway, 17th Floor New York, NY USA: ACM Press, 1994. 129~138
[39]王变琴.日志管理技术研究与实现.计算机工程, 2000, 26(8): 115~116
[40]邹永贵.数据库恢复系统的设计.重庆邮电学院学报, 1998, 10(2): 36~39
[41]贾焰,王志英,韩伟红等.分布式数据库技术.第一版.北京:国防工业出版社, 2000. 87~111
[2] R. Agrawal, D. DeWitt. Integrated Concurrency Control and Recovery Mechanisms: Design and Performance Evaluation. ACM Transactions on Database Systems, 1985, 10(4): 529~564
[3] Andreas Reuter. Performance Analysis of Recovery Techniques. ACM Transactions on Database Systems, 1984, 9(4): 526~559
[4] J. S. Keen, W. J. Dally. Performance Evaluation of Ephemeral Logging. in: Peter Buneman, Sushil Jajodia eds.. Preceedings of the 1993 ACM SIGMOD International Conference on Management of Data. Washington, D.C., United States. 1993. 1515 Broadway, 17th Floor New York, NY USA: ACM Press, 1993. 187~196
[5] Richard Dean Regan. High Performance Ephermeral Logging for Database Systems: [Dissertation for the Degree of Doctor of Philosophy (Computer Science)]. Michigan 48106-1346 USA: Polytechnic University, 2001
[6] J. S. Keen, W. J. Dally. XEL: Extended Ephemeral Logging for Log Storage Managerment. in:N. R. Adam, B. K. Bhargava, Yelena Yesha eds.. Proceedings of the Third International Conference on Information and Knowledge Management. Gaithersburg, Maryland, United States. 1994. 1515 Broadway, 17th Floor New York, NY USA: ACM Press, 1994. 312~321
[7] J. S. Keen, W. J. Dally. Extended Ephemeral Logging: Log Storage Management for Application with Long Lived Transactions. ACM Transactions on Database Systems, 1997, 22(1): 1~42
[8]宋广华,杨长生.基于混合日志的内存数据库恢复子系统.浙江大学学报(理学报), 2001, 28(2): 164~168
[9] Taesoon Park, Junguk L. Kim. A Checkpointing Recovery Approach in A Distributed System on the CSMA/CD Network. in: Hal Berghel, Ed Deaton, George Hedrick, David Roach, Roger Wainwright eds.. Proceedings of the 1992 ACM/SIGAPP Symposium on Applied Computing. Kansas City, Missouri, United States. 1992. 1515 Broadway, 17th Floor New York, NY USA: ACM Press, 1992. 614~623
[10] Raj Sekhar Pamnla, P. K. Srimeni. Check Pointing Strategies for Database Systems. in: Pat Davis, Vicki McClintock eds.. Proceedings of the 15th Annual Conference on Computer Science. St. Louis, Missouri, United States, 1987. 1515 Broadway, 17th Floor New York, NY USA: ACM Press, 1987. 88~97
[11] Theo Haerder, Andreas Reuter. Principles of Transaction Oriented Database Recovery. Computing Surveys, 1983, 15(4): 287~317
[12]于秀云.对数据库恢复策略的性能分析.计算机研究与发展, 1998, 35(12): 1135~1141
[13] J. Gray, P. McJones, M. Blasgen, et al. The Recovery Manager of the System R Database Manager. ACM Computing Surveys, 1981, 13(2): 223~243
[14] Hector Garcia Molina, J. D. Ullman, Jennifer Widom.数据库系统实现.第一版.杨冬青,唐世渭,徐其钧.北京:机械工业出版社, 2002. 304~331
[15] A. Reuter. A Fast Transaction-Oriented Logging Scheme for Undo Recovery. IEEE Transactions on Software Engineering, 1980, 6(4): 348~356
[16] David Lomet, Mark Tuttle. A Theory of Redo Recovery. in: Yannis Papakonstantinou, Alon Halevy, Zachary Ives eds.. Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data. San Diego, California. 2003. 1515 Broadway, 17th Floor New York, NY USA: ACM Press, 2003. 397~406
[17] C. Mohan, D. Haderle, B. Lindsay, et al. ARIES: A Transaction Recovery MethodSupporting Fine-Granularity Locking and Partial Rollbacks Using Write-Ahead Logging. ACM Transactions on Database Systems, 1992, 17(1): 94~162
[18]施晓秋. ARIES数据库恢复算法的分析.温州大学学报, 2000, 37(2): 46~50
[19]陈宇,孟煊,王燕. DB2数据库恢复策略及实现.西南民族学院学报(自然科学版), 1999, 25(1): 26~29
[20] Rama Velpufi, Anand Adkoli. Oracle8i备份与恢复手册.第一版.蒋蕊,王磊,王毳等.北京:机械工业出版社, 2001. 117~252
[21] Dean Kuo. Model and Verification of A Data Manager Based on ARIES. ACM Transactions on Database Systems (TODS), 1996, 21(4): 427~479
[22]王意洁,胡守仁.一种实用有效的恢复策略WAL_P.国防科技大学学报, 2000, 22(1): 7~10
[23] C. Mohan, I. Narang. ARIES/CSA: A Method for Database Recovery in Client-Server Architechtures. ACM SIGMOD Record, 1994, 23(2): 55~66
[24]黄志华. Oracle的故障恢复机制.漳州师范学院学报(自然科学版), 2002, 15(3): 29~32
[25]冯玉才.数据库系统基础.第二版.武汉:华中理工大学出版社, 1993. 29~30
[26]曾利平,刘新华. Oracle9i数据库的备份与恢复.计算机与现代化, 2002, 3(12): 62~64
[27]郭鲜凤,张和义. SQL Server分部式数据库的并发控制和故障恢复.科技情报开发与经济, 2002, 12(1): 60~61
[28] Raghu Ramakrishnan, Johannes Gehrke.数据库管理系统.第二版.周立柱,蒋旭东,石晶等.北京:清华大学出版社, 2002. 431~445
[29]阳富民,冯玉才,吴恒山等.一种分部式DBMS故障恢复技术.计算机研究与发展, 1995, 32(5): 25~30
[30]许蔚.分部式数据库DM3并发控制和故障恢复研究:[硕士学位论文]。武汉:华中理工大学图书馆,2000
[31] Vassos Hadzilacos. A Theory of Reliability in Database Systems. Journal of the Association for Computing Machinery, 1988, 35(1): 121~145
[32] Michael Stone Braker. Readings in Database Systems. First Edition. 340 Pine Street Sixth Floor San Francisco, CA USA: Morgan Kaufmann Publisher Inc, 1988. 140~150
[33]萨师渲,王珊.数据库系统概论.第三版.北京:高等教育出版社, 2000. 247~262
[34] Heiko Schuldt, Gustavo Alonso, Hans-Jorg Schek. Concurrency Control and Recovery in Transactional Process Management. in: Victor Vianu, Christors Papadimitriou eds.. Proceedings of the Eighteenth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems. Philadelphia, Pennsylvania, United States. 1999. 1515 Broadway, 17th Floor New York, NY USA: ACM Press, 1999. 316~326
[35] Abraham Silberschatz, H. F. Korth, S. Sudarshan.数据库系统概念.第三版.杨冬青,唐世渭.北京:机械工业出版社, 1999. 359~379
[36] D. B. Lomet. High Speed On-line Backup when Using Logical Log Operations. in: Maggie Dunham, J. F. Naughtom, Weidong Chen, Nick Koudas eds.. International Conference on Management of Data and Symposium on Principles of Database Systems. Dallas, Texas, United States. 2000. 1515 Broadway, 17th Floor New York, NY USA: ACM Press, 2000. 34~45
[37] David Lomet, Mark Tuttle. Logical Logging to Extend Recovery to New Domains. in: Susan B. Davidson, Christos Faloutsos eds.. Proceedings of the 1999 ACM SIGMOD International Conference on Management of Data. Philadelphia, Pennsylvania, United States. 1999. 1515 Broadway, 17th Floor New York, NY USA: ACM Press, 1999. 73~84
[38] G. Alonso, D. Agrawal, A. E. Abbadi. Reducing Recovery Constraints on Locking Based Protocols. in: Victor Vianu ed. Proceedings of the Thirteenth ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems. Minneapolis, Minnesota, United States. 1994. 1515 Broadway, 17th Floor New York, NY USA: ACM Press, 1994. 129~138
[39]王变琴.日志管理技术研究与实现.计算机工程, 2000, 26(8): 115~116
[40]邹永贵.数据库恢复系统的设计.重庆邮电学院学报, 1998, 10(2): 36~39
[41]贾焰,王志英,韩伟红等.分布式数据库技术.第一版.北京:国防工业出版社, 2000. 87~111