服务器集群系统高可用性研究及管理软件的实现
|
摘要
随着通过Internet提供商业服务的发展，服务系统的可用性程度对服务提供者的商业利益具有重大的影响。同时，由于通过计算机服务系统提供的服务内容和服务范围都在不断扩大，计算机服务系统的规模也需不断扩大，现有的小规模的高可用系统已经难以满足这样大规模计算机系统对高可用支持的需求。因此研究可扩展的高可用集群系统十分重要。
集群系统的优点是可扩展性好，但随着集群系统规模的增大，结点数目的增多，集群系统整体的可靠性会相应降低。另外，用于支持不同应用管理的服务器集群系统涉及的用户数目庞大，因此高可用性对服务器集群系统具有格外的重要意义。提高集群系统可用性的软件将成为集群操作系统中必不可少的部分，特别是故障监测恢复和集群系统的管理手段对大规模系统和长时间运行的应用显得尤为重要。可以说，集群的体系结构为集群高可用研究提供了可能性，服务器集群系统在软件方面提出集群高可用研究的必要性，高可用性技术和集群技术的结合是近年来计算机领域的一个重要研究方向。
本文的目的就是研究服务器集群系统的高可用性。具体的方法是首先分析集群系统的结构特征以及服务器集群软件应具备的性能，然后在分析高可用系统理论及关键技术的基础上，推倒出并行结构与串行结构高可用性公式。对双机高可用系统进行研究与改进，最后给出了双机高可用系统可用性的估算值，并测试了双机高可用系统的切换时间。针对后端服务器，我们将软件再生技术引入集群高可用系统中，设计并实现了一个作为基于集中式集群结构的服务器软件之一的集群高可用管理系统，给出了几种软件老化模型和算法并提出一种新的算法，并就实现中的关键问题进行了详细的描述。最后，我们对集群高可用系统做出了可用性评价。
With the explosive growth of commercial service via Internet, High Availability of server system is strongly required for the benefits of servers. In addition, the current server systems have to be scaled to meet the development of service content and the diversity of service. The practical high availability system in use cannot satisfy the scalable server system, so it is important to do some research on scalable, high-available cluster system.
The advantage of cluster system is scalability. While the nodes in cluster become larger, the reliability of the whole system will be lower. The clients of different services on server cluster system are so big that high availability means much of server cluster. High-availability software is an essential part of cluster system. Fault detection, fault recovery and management software is especially important for large scale and long running application. The framework of cluster makes high availability possible while server cluster system make high availability essential. Research on combination of high availability and cluster is an important branch of computer field in recent years.
In this paper we review the state of the art techniques including server cluster's architecture and required performance of server cluster software. Then based on the fundamentals of high availability and key techniques, we do some research and improvement on double dispatcher high availability system. Then we induce a new technique named software rejuvenation in cluster high availability system, implement a high availability management system, give some models and algorithms of software aging and proposed a new algorithm. At last, we give an evaluation of cluster high availability system.
集群系统的优点是可扩展性好，但随着集群系统规模的增大，结点数目的增多，集群系统整体的可靠性会相应降低。另外，用于支持不同应用管理的服务器集群系统涉及的用户数目庞大，因此高可用性对服务器集群系统具有格外的重要意义。提高集群系统可用性的软件将成为集群操作系统中必不可少的部分，特别是故障监测恢复和集群系统的管理手段对大规模系统和长时间运行的应用显得尤为重要。可以说，集群的体系结构为集群高可用研究提供了可能性，服务器集群系统在软件方面提出集群高可用研究的必要性，高可用性技术和集群技术的结合是近年来计算机领域的一个重要研究方向。
本文的目的就是研究服务器集群系统的高可用性。具体的方法是首先分析集群系统的结构特征以及服务器集群软件应具备的性能，然后在分析高可用系统理论及关键技术的基础上，推倒出并行结构与串行结构高可用性公式。对双机高可用系统进行研究与改进，最后给出了双机高可用系统可用性的估算值，并测试了双机高可用系统的切换时间。针对后端服务器，我们将软件再生技术引入集群高可用系统中，设计并实现了一个作为基于集中式集群结构的服务器软件之一的集群高可用管理系统，给出了几种软件老化模型和算法并提出一种新的算法，并就实现中的关键问题进行了详细的描述。最后，我们对集群高可用系统做出了可用性评价。
With the explosive growth of commercial service via Internet, High Availability of server system is strongly required for the benefits of servers. In addition, the current server systems have to be scaled to meet the development of service content and the diversity of service. The practical high availability system in use cannot satisfy the scalable server system, so it is important to do some research on scalable, high-available cluster system.
The advantage of cluster system is scalability. While the nodes in cluster become larger, the reliability of the whole system will be lower. The clients of different services on server cluster system are so big that high availability means much of server cluster. High-availability software is an essential part of cluster system. Fault detection, fault recovery and management software is especially important for large scale and long running application. The framework of cluster makes high availability possible while server cluster system make high availability essential. Research on combination of high availability and cluster is an important branch of computer field in recent years.
In this paper we review the state of the art techniques including server cluster's architecture and required performance of server cluster software. Then based on the fundamentals of high availability and key techniques, we do some research and improvement on double dispatcher high availability system. Then we induce a new technique named software rejuvenation in cluster high availability system, implement a high availability management system, give some models and algorithms of software aging and proposed a new algorithm. At last, we give an evaluation of cluster high availability system.
引文
[1] http://www-900.ibm.com/developerWorks/cn/linux/cluster/lvs/part/index.shtml Linux服务器集群系统(一)——LVS项目介绍作者：章文嵩
[2] D.M.Dias, W. Kish, R. Mukherjee, and R. Tewari. A scalable and highly available web server. In Proceedings of IEEE COMPCON'96, pages 85-92, 1996
[3] D. Andresen, T. Yang, V. Holmedahl, and O. H. Ibarra. SWEB: Toward a Scalable World Wide Web Server on Multi-computers. In Proceedings of the 10th International Symposium on Parallel Processing (ISPP'96), Apr. 1996, 850-856
[4] Xiaolan Zhang, Michael Barrientos, J.Bradley Chen, HACC: An Architecture for Cluster-Based Web Servers. Appear in the 3rd USENIX Windows NT Symposium
[5] R. Chillarege, S. Biyani, and Rosenthal. Measurement of Failure Rate in Widely Distributed Software. In Proc. Of 25th IEEE Intl. Symposium on Fault Tolerant Computing, page 424-433, Pasadena, CA, July 1995.
[6] A. Thakur and R. K. Iyer. Analyze-Now-An Environment for Collection and Analysis of Failures in a Network of Workstations. In Intl. Symposium on Software Reliablity Engineering, page 14-23, White Plains, NY, April 1996.
[7] mosix available on http://www.redcluster.net/mosix/index.htm
[8] Luis Aversa and Azer Bestavros. Load Balancing A Cluster Of Web Servers Using Distributed Packet Rewriting.In Proceedings of the IEEE International Performance, Computing and Communications conference, Phoenix, AZ USA, February 2000.C.Perkins, IETF RFC2003
[9] J. Gray and D. P. Siewiorek, "High-Availability Computer Systems," IEEE Computer 24, 39-48 (September 1991).
[10] E. Anderson, D. Patterson, and E. Brewer. The MagicRouter: An application of fast packet interposing. http://www.cs.berkely.edu/eanders/projects/magicrouter/osdi96-mr-submisson.ps, May 1996
[11] Stephen T. Satchell, H. B. J Clifford．刘隆国等译．Linux IP协议栈源代码分析，机械工业出版社，2000
[12] 《用TCP／IP进行网际互联第一卷：原理、协议和结构》赵刚等译电子工业出版社
[13] Y. Huang, C. Kintala, N. Kolettis, and N. D. Fulton. Software Rejuvenation: Analysis,Module and Applications[J]. In Proc. Of 25th Symposium on Fault Tolerant Computer Systems, pages 381-390, Pasadena, California, June 1995.
[14] Y. Huang, P. Jalote, and C. Kintala. Lecture Notes in Computer Science, Vol. 774, chapter Two techniques for transient software error recovery[C]. pages 159-170. Springer Verlag, Berlin, 1994.
[15] M.F. Arlitt and T. Jin. A workload characterization of the 1998 world cup web site. IEEE Network, 14(3):30-37, May/June 2000.
[16] M.F. Arlitt and C. L. Williamson. Internet web servers: Workload characterization and performance implications.IEEE/ACM Transactions on Networking, 5(5):631-645,Oct. 1997.
[17] V.Castelli, R. E. Harper, P. Heidelberger, S. W. Hunter, K.S. Trivedi, K. Vaidyanathan and W. Zeggert. Proactive Management of Software Aging[J]. IBM Journal of Research & Development, Vol. 45, No.2, March 2001.
[18] J.A. Hartigan. Clustering Algorithms[M]. New York:Wiley, 1975.
[19] M.C. Hsueh, R. K. Iyer, and K. S. Trivedi. Performability Modeling Based on Real Data: A Case Study[C]. IEEE Transactions on Computers, 37(4):478-484, April 1988
[20] P.K. Sen. Estimates of the Regression Coefficient Based on Kendall's Tau[J]. Journal of the American Statistical association, 63:1379-1389, 1968.
[21] S. Garg, A. van Moorsel, K. Vaidyanathan, K. Trivedi. A Methodology for Detection and Estimation of Software Aging[C]. In Proc. of 9th Intnl. Symposium on Software Reliability Engineering, pages 282-292, Paderborn, Germany, November 1998.
[22] K. Vaidyanathan and K. S. Trivedi, "A Measurement-Based Model for Estimation of Resource Exhaustion in Operational Software Systems," Proceedings of the Tenth IEEE International Symposium on Software
Reliability Engineering, Boca Raton, FL, November 1999, pp. 84-93.
[23] D. Tang and R. K. Iyer. Dependability Measurement Modeling of a Multicomputer System. IEEE Transactions on Computers, 42(1), January 1993.
[24] Kurt Wall，张辉．GUN／Linux编程指南入门·应用·精通[M]．北京：清华大学出版社，2002．
[25] W. Richard Stevens，施振川．UNIX网络编程第一卷 套接口，API和X/OPEN传输接口，API(第2版)[M]．北京：清华大学出版社．2001．
[26] 王文义，辛小南，王若雨．高性能集群计算机的构建与应用[J]．计算机工程与应用，2001，37(17)：157—159
[27] R.K. Iyer and D. J. Rossetti, "Effect of System Workload on Operating System Reliability: A Study on IBM 3081." IEEE Trans. Software Eng. SE-11, No. 12, 1438-1448 (December 1985).
[28] S. Garg, Y. Huang, C. Kintala, and K. S. Trivedi, "Minimizing Completion Time of a Program by Checkpointing and Rejuvenation," Proceedings of the 1996 ACM SIGMETRICS Conference, Philadelphia, PA, May 1996, pp. 252-261.
[29] S. Garg, A. Puliafito, and K. S. Trivedi, "Analysis of Software Rejuvenation Using Markov Regenerative Stochastic Petri Net," Proceedings of the Sixth International Symposium on Software Reliability Engineering, Toulouse, France, October 1995, pp. 180-187.
[30] C. Gage, IBM Secureway Network Dispatcher 2.1;http://www-4.ibm.com/software/network/dispatcher/library/.
[31] M.R. Lyu and V. B. Mendiratta, "Software Fault Tolerance in a Clustered Architecture: Techniques and Reliability Modeling," Proceedings of the 1999 IEEE Aerospace Conference, Snowmass, CO, March 1999, pp. 141-150.
[32] S.W. Hunter and W. E. Smith, "Availability Modeling and Analysis of a Two Node Cluster," Proceedings of the 5th International Conference on Information Systems, Analysis and Synthesis, Orlando, FL, October 1999.
[33] S. Garg, A. Puliafito, M. Telek, and K. S. Trivedi, "Analysis of Preventive Maintenance in Transactions Based Software Systems," IEEE Trans. Computers 47, No. 1, 96-107 (January 1998).
[34] Kai Hwang．高级计算机体系结构(英文版)，机械工业出版社，1999
[35] 王仲生着．智能容错技术及应用，国防工业出版社，2002．9
[36] W. Vogels, "The Design and Architecture of the Microsoft Cluster Service-A Practical Approach to High-Availability and Scalability," Proc. 28th Symposium on Fault-Tolerant Computing, CS Press, 1998, pp. 422-431.
[37] E.G. Coffman and E. N. Gilbert. Optimal strategies for scheduling checkpoints and preventive maintenance. IEEE Transactions on Reliability, 39(1):9-18, April 1990.
[38] 《Linux网络编程》林宇、郭凌云等 人民邮电出版社 2000
[39] Httperf Available online at http://ftp.hpl.com.cn/pub/httperf
[40] R.K. Iyer and D. J. Rossetti, "Effect of System Workload on Operating System Reliability: A Study on IBM 3081," IEEE Trans. Software Eng. SE-11, No. 12, 1438-1448 (December 1985).
[41] M.R. Lyu and V. B. Mendiratta, "Software Fault Tolerance in a Clustered Architecture: Techniques and Reliability Modeling," Proceedings of the 1999 IEEE Aerospace Conference, Snowmass, CO. March 1999, pp. 141-150.
[42] S. Garg, A. van Moorsel, K. Vaidyanathan, and K. Trivedi, "A Methodology for Detection and Estimation of Software Aging," Proceedings of the 9th International Symposium on Software Reliability Engineering, Paderborn, Germany, November 1998, pp. 282-292.
[43] 高文。服务器聚集系统中高可用性分析与设计方法。[博士论文]中国科学院计算技术研究所2001
[44] 郑伟，郭成城。一种提高集群系统可用性的方法及其应用。计算机应用。2003．10 P67—70
[45] Bobbio. A.: Garg, S.; Gribaudo, M.; Sereno, M.; Horvath, A.; Telek, M. "Modeling software systems with rejuvenation, restoration and checkpointing through fluid stochastic Petri nets". Proc. 8th Int. Workshop on Petri Net and Performance Models (PNPM'99), 8-10 October 1999, Zaragoza, Spain, pages 82-91. 1999.
[46] C.S. Yang and M.Y. Luo. "Efficient Support for Content-Based Routing in Web Server Clusters." In Proceedings of the 2nd Usenix Symposium on Internet Technologies and Systems, Boulder, CO, Oct. 1999.
[47] N. Aghdaie and Y. Tamir. Implementation and evaluation of transparent fault-tolerant web service with kernel-level support. In Proc. IEEE Intl. Conf. on Computer Communications and Networks, 2002.
[48] N. Aghdaie and Y. Tamir, "Client-Transparent Fault-Tolerant Web Service," Proceedings of the 20th IEEE International Performance, Computing, and Communications Conference, Phoenix, Arizona, pp. 209-216 (April 2001).
[49] S. Garg, A. van Moorsel, K. Vaidyanathan, K. Trivedi. A Methodology for Detection and Estimation of Software Aging. In Proc. of 9th Intl. Symposium on Software Reliability Engineering, pages 282-292, Paderborn, Germany, November 1998.
[50] K. Vaidyanathan and K. S. Trivedi. A Measurement-Based Model for Estimation of Resource Exhaustion in Operational Software Systems. In Proc. of the Tenth IEEE Intl. Symposium on Software Reliability Engineering, pages 84-93, Boca Raton, Florida, November 1999.
[51] W. Vogels. The Design and Architecture of the Microsoft Cluster Service - A Practical Approach to High-Availability and Scalability. lnProc. 28th IEEE Intl. Symp. on Fault-Tolerant Computing, pages 422-431, 1998.
[52] http://parapet.ee.princeton.edu/～sigm2002/rej uvenation.html Software Rejuvenation- Modeling and Analysis
[53] Eric Anderson, Dave Patterson, and Erie Brewer, "The Magicrouter: an application of Fast Packet Interposing", http://www.cs.berkeley.edu/～eanders-/magicrouter/, May, 1996.
[54] 《智能容错技术及应用》 王仲生 着国防工业出版社 2002．9
[55] 基于Linux的集群系统-关键技术分析之高可用性 http://www-900.ibm.com/developerWorks/cn/linux/cluster/linux_cluster/pa rt6/index.shtml
[56] The Linux Virtual Server Project. http://www.linuxvirtualserver.org/.
[57] MON available on http://linux-ha.org
[2] D.M.Dias, W. Kish, R. Mukherjee, and R. Tewari. A scalable and highly available web server. In Proceedings of IEEE COMPCON'96, pages 85-92, 1996
[3] D. Andresen, T. Yang, V. Holmedahl, and O. H. Ibarra. SWEB: Toward a Scalable World Wide Web Server on Multi-computers. In Proceedings of the 10th International Symposium on Parallel Processing (ISPP'96), Apr. 1996, 850-856
[4] Xiaolan Zhang, Michael Barrientos, J.Bradley Chen, HACC: An Architecture for Cluster-Based Web Servers. Appear in the 3rd USENIX Windows NT Symposium
[5] R. Chillarege, S. Biyani, and Rosenthal. Measurement of Failure Rate in Widely Distributed Software. In Proc. Of 25th IEEE Intl. Symposium on Fault Tolerant Computing, page 424-433, Pasadena, CA, July 1995.
[6] A. Thakur and R. K. Iyer. Analyze-Now-An Environment for Collection and Analysis of Failures in a Network of Workstations. In Intl. Symposium on Software Reliablity Engineering, page 14-23, White Plains, NY, April 1996.
[7] mosix available on http://www.redcluster.net/mosix/index.htm
[8] Luis Aversa and Azer Bestavros. Load Balancing A Cluster Of Web Servers Using Distributed Packet Rewriting.In Proceedings of the IEEE International Performance, Computing and Communications conference, Phoenix, AZ USA, February 2000.C.Perkins, IETF RFC2003
[9] J. Gray and D. P. Siewiorek, "High-Availability Computer Systems," IEEE Computer 24, 39-48 (September 1991).
[10] E. Anderson, D. Patterson, and E. Brewer. The MagicRouter: An application of fast packet interposing. http://www.cs.berkely.edu/eanders/projects/magicrouter/osdi96-mr-submisson.ps, May 1996
[11] Stephen T. Satchell, H. B. J Clifford．刘隆国等译．Linux IP协议栈源代码分析，机械工业出版社，2000
[12] 《用TCP／IP进行网际互联第一卷：原理、协议和结构》赵刚等译电子工业出版社
[13] Y. Huang, C. Kintala, N. Kolettis, and N. D. Fulton. Software Rejuvenation: Analysis,Module and Applications[J]. In Proc. Of 25th Symposium on Fault Tolerant Computer Systems, pages 381-390, Pasadena, California, June 1995.
[14] Y. Huang, P. Jalote, and C. Kintala. Lecture Notes in Computer Science, Vol. 774, chapter Two techniques for transient software error recovery[C]. pages 159-170. Springer Verlag, Berlin, 1994.
[15] M.F. Arlitt and T. Jin. A workload characterization of the 1998 world cup web site. IEEE Network, 14(3):30-37, May/June 2000.
[16] M.F. Arlitt and C. L. Williamson. Internet web servers: Workload characterization and performance implications.IEEE/ACM Transactions on Networking, 5(5):631-645,Oct. 1997.
[17] V.Castelli, R. E. Harper, P. Heidelberger, S. W. Hunter, K.S. Trivedi, K. Vaidyanathan and W. Zeggert. Proactive Management of Software Aging[J]. IBM Journal of Research & Development, Vol. 45, No.2, March 2001.
[18] J.A. Hartigan. Clustering Algorithms[M]. New York:Wiley, 1975.
[19] M.C. Hsueh, R. K. Iyer, and K. S. Trivedi. Performability Modeling Based on Real Data: A Case Study[C]. IEEE Transactions on Computers, 37(4):478-484, April 1988
[20] P.K. Sen. Estimates of the Regression Coefficient Based on Kendall's Tau[J]. Journal of the American Statistical association, 63:1379-1389, 1968.
[21] S. Garg, A. van Moorsel, K. Vaidyanathan, K. Trivedi. A Methodology for Detection and Estimation of Software Aging[C]. In Proc. of 9th Intnl. Symposium on Software Reliability Engineering, pages 282-292, Paderborn, Germany, November 1998.
[22] K. Vaidyanathan and K. S. Trivedi, "A Measurement-Based Model for Estimation of Resource Exhaustion in Operational Software Systems," Proceedings of the Tenth IEEE International Symposium on Software
Reliability Engineering, Boca Raton, FL, November 1999, pp. 84-93.
[23] D. Tang and R. K. Iyer. Dependability Measurement Modeling of a Multicomputer System. IEEE Transactions on Computers, 42(1), January 1993.
[24] Kurt Wall，张辉．GUN／Linux编程指南入门·应用·精通[M]．北京：清华大学出版社，2002．
[25] W. Richard Stevens，施振川．UNIX网络编程第一卷 套接口，API和X/OPEN传输接口，API(第2版)[M]．北京：清华大学出版社．2001．
[26] 王文义，辛小南，王若雨．高性能集群计算机的构建与应用[J]．计算机工程与应用，2001，37(17)：157—159
[27] R.K. Iyer and D. J. Rossetti, "Effect of System Workload on Operating System Reliability: A Study on IBM 3081." IEEE Trans. Software Eng. SE-11, No. 12, 1438-1448 (December 1985).
[28] S. Garg, Y. Huang, C. Kintala, and K. S. Trivedi, "Minimizing Completion Time of a Program by Checkpointing and Rejuvenation," Proceedings of the 1996 ACM SIGMETRICS Conference, Philadelphia, PA, May 1996, pp. 252-261.
[29] S. Garg, A. Puliafito, and K. S. Trivedi, "Analysis of Software Rejuvenation Using Markov Regenerative Stochastic Petri Net," Proceedings of the Sixth International Symposium on Software Reliability Engineering, Toulouse, France, October 1995, pp. 180-187.
[30] C. Gage, IBM Secureway Network Dispatcher 2.1;http://www-4.ibm.com/software/network/dispatcher/library/.
[31] M.R. Lyu and V. B. Mendiratta, "Software Fault Tolerance in a Clustered Architecture: Techniques and Reliability Modeling," Proceedings of the 1999 IEEE Aerospace Conference, Snowmass, CO, March 1999, pp. 141-150.
[32] S.W. Hunter and W. E. Smith, "Availability Modeling and Analysis of a Two Node Cluster," Proceedings of the 5th International Conference on Information Systems, Analysis and Synthesis, Orlando, FL, October 1999.
[33] S. Garg, A. Puliafito, M. Telek, and K. S. Trivedi, "Analysis of Preventive Maintenance in Transactions Based Software Systems," IEEE Trans. Computers 47, No. 1, 96-107 (January 1998).
[34] Kai Hwang．高级计算机体系结构(英文版)，机械工业出版社，1999
[35] 王仲生着．智能容错技术及应用，国防工业出版社，2002．9
[36] W. Vogels, "The Design and Architecture of the Microsoft Cluster Service-A Practical Approach to High-Availability and Scalability," Proc. 28th Symposium on Fault-Tolerant Computing, CS Press, 1998, pp. 422-431.
[37] E.G. Coffman and E. N. Gilbert. Optimal strategies for scheduling checkpoints and preventive maintenance. IEEE Transactions on Reliability, 39(1):9-18, April 1990.
[38] 《Linux网络编程》林宇、郭凌云等 人民邮电出版社 2000
[39] Httperf Available online at http://ftp.hpl.com.cn/pub/httperf
[40] R.K. Iyer and D. J. Rossetti, "Effect of System Workload on Operating System Reliability: A Study on IBM 3081," IEEE Trans. Software Eng. SE-11, No. 12, 1438-1448 (December 1985).
[41] M.R. Lyu and V. B. Mendiratta, "Software Fault Tolerance in a Clustered Architecture: Techniques and Reliability Modeling," Proceedings of the 1999 IEEE Aerospace Conference, Snowmass, CO. March 1999, pp. 141-150.
[42] S. Garg, A. van Moorsel, K. Vaidyanathan, and K. Trivedi, "A Methodology for Detection and Estimation of Software Aging," Proceedings of the 9th International Symposium on Software Reliability Engineering, Paderborn, Germany, November 1998, pp. 282-292.
[43] 高文。服务器聚集系统中高可用性分析与设计方法。[博士论文]中国科学院计算技术研究所2001
[44] 郑伟，郭成城。一种提高集群系统可用性的方法及其应用。计算机应用。2003．10 P67—70
[45] Bobbio. A.: Garg, S.; Gribaudo, M.; Sereno, M.; Horvath, A.; Telek, M. "Modeling software systems with rejuvenation, restoration and checkpointing through fluid stochastic Petri nets". Proc. 8th Int. Workshop on Petri Net and Performance Models (PNPM'99), 8-10 October 1999, Zaragoza, Spain, pages 82-91. 1999.
[46] C.S. Yang and M.Y. Luo. "Efficient Support for Content-Based Routing in Web Server Clusters." In Proceedings of the 2nd Usenix Symposium on Internet Technologies and Systems, Boulder, CO, Oct. 1999.
[47] N. Aghdaie and Y. Tamir. Implementation and evaluation of transparent fault-tolerant web service with kernel-level support. In Proc. IEEE Intl. Conf. on Computer Communications and Networks, 2002.
[48] N. Aghdaie and Y. Tamir, "Client-Transparent Fault-Tolerant Web Service," Proceedings of the 20th IEEE International Performance, Computing, and Communications Conference, Phoenix, Arizona, pp. 209-216 (April 2001).
[49] S. Garg, A. van Moorsel, K. Vaidyanathan, K. Trivedi. A Methodology for Detection and Estimation of Software Aging. In Proc. of 9th Intl. Symposium on Software Reliability Engineering, pages 282-292, Paderborn, Germany, November 1998.
[50] K. Vaidyanathan and K. S. Trivedi. A Measurement-Based Model for Estimation of Resource Exhaustion in Operational Software Systems. In Proc. of the Tenth IEEE Intl. Symposium on Software Reliability Engineering, pages 84-93, Boca Raton, Florida, November 1999.
[51] W. Vogels. The Design and Architecture of the Microsoft Cluster Service - A Practical Approach to High-Availability and Scalability. lnProc. 28th IEEE Intl. Symp. on Fault-Tolerant Computing, pages 422-431, 1998.
[52] http://parapet.ee.princeton.edu/～sigm2002/rej uvenation.html Software Rejuvenation- Modeling and Analysis
[53] Eric Anderson, Dave Patterson, and Erie Brewer, "The Magicrouter: an application of Fast Packet Interposing", http://www.cs.berkeley.edu/～eanders-/magicrouter/, May, 1996.
[54] 《智能容错技术及应用》 王仲生 着国防工业出版社 2002．9
[55] 基于Linux的集群系统-关键技术分析之高可用性 http://www-900.ibm.com/developerWorks/cn/linux/cluster/linux_cluster/pa rt6/index.shtml
[56] The Linux Virtual Server Project. http://www.linuxvirtualserver.org/.
[57] MON available on http://linux-ha.org