高性能服务器自主管理板的设计与实现
|
摘要
高性能、高可用服务器,可以应用于电信、金融、工业、能源、及政府部门等多个涉及国家安全及国计民生的领域。而目前国内对高可用的研究主要还是通过冗余技术实现,造成较大的冗余度,因此研制具备自适应、可扩展、可重构、自配置能力的高性能、高可用服务器,能够完善我国信息化建设,对国家的经济、社会安全有着战略性的意义。
而服务器管理是研制高性能、高可用服务器的关键,为了实现对整个服务器集中、持续、有效地管理,设计了高性能服务器管理硬件平台。为了实现高可用性,采用多层次架构,设计了本地管理模块LMM和全局管理模块GMM。LMM主要负责对本地节点的状态收集和分析及故障的检测和处理,GMM主要负责监控和管理整个系统,它本身并不直接监测计算节点的软硬件资源运行状态。为了实现对服务器系统内计算节点运行状态的全面监控,采用了带内外相结合的监控方式,实现对服务器系统内部件运行状态的实时监测。此外,GMM的双机热备份技术,多通道的通讯网络冗余技术,保证了系统的高可用。热插拔功能的设计更能明显缩短管理系统的平均维修时间,进一步提高系统的可用性。
本文研究了基于IPMI的服务器管理技术、常见的高可用技术,根据高性能、高可用服务器的需求,总结了服务器管理平台的功能需求,然后针对此需求提出了LMM和GMM双层管理架构,并完成了对LMM和GMM的详细设计。最后采用ARM7最小系统+外围通讯接口群的嵌入式系统完成了LMM的硬件实现,并对板卡各个模块及主要功能进行了测试。
High-performance and high availability servers can be used in many fields which involved in national security and people's livelihood such as telecommunications and so on. Nowadays, the research of domestic on availability is redundancy, resulting in greater redundancy.So developping the high-performance and high availability servers with adaptive, scalable, reconfigurable and self-configuration capabilities can improve information construction in China,and have a strategic significance on the country's economy and social security.
Server management is the key technology of developing high-performance and high availability server. In order to achieve the focused, sustained and effective management of server, the paper designs a hardware platform for high-performance server management.The design uses multi-level architecture to achieve high availability-LMM(local management module) and GMM(global management module). LMM is mainly responsible for the state of the local node, GMM is responsible for monitoring and managing the entire system. The design using a combination of in-band and out-of-band monitoring ways to monitor the compute nodes in real-time. In addition, GMM’s hot backup technology, and multi-channel communication network redundancy ensures system availability. Hot-plug feature is designed to shorter mean maintenance time, so as to improve system availability.
The paper studys the server management technology based on IPMI , the common high-availability technologies, and the needs of the high-performance, high availability server .The paper summarizes the functional requirements of server management platform and then puts forward a double-layer management structure-- LMM and GMM, the detailed design of LMM and GMM is finished at the same time . Finally the hardware of LMM is implemented using a ARM7 minimum system + Peripheral Communication Interface Group , and the main functions of each module is tested.
而服务器管理是研制高性能、高可用服务器的关键,为了实现对整个服务器集中、持续、有效地管理,设计了高性能服务器管理硬件平台。为了实现高可用性,采用多层次架构,设计了本地管理模块LMM和全局管理模块GMM。LMM主要负责对本地节点的状态收集和分析及故障的检测和处理,GMM主要负责监控和管理整个系统,它本身并不直接监测计算节点的软硬件资源运行状态。为了实现对服务器系统内计算节点运行状态的全面监控,采用了带内外相结合的监控方式,实现对服务器系统内部件运行状态的实时监测。此外,GMM的双机热备份技术,多通道的通讯网络冗余技术,保证了系统的高可用。热插拔功能的设计更能明显缩短管理系统的平均维修时间,进一步提高系统的可用性。
本文研究了基于IPMI的服务器管理技术、常见的高可用技术,根据高性能、高可用服务器的需求,总结了服务器管理平台的功能需求,然后针对此需求提出了LMM和GMM双层管理架构,并完成了对LMM和GMM的详细设计。最后采用ARM7最小系统+外围通讯接口群的嵌入式系统完成了LMM的硬件实现,并对板卡各个模块及主要功能进行了测试。
High-performance and high availability servers can be used in many fields which involved in national security and people's livelihood such as telecommunications and so on. Nowadays, the research of domestic on availability is redundancy, resulting in greater redundancy.So developping the high-performance and high availability servers with adaptive, scalable, reconfigurable and self-configuration capabilities can improve information construction in China,and have a strategic significance on the country's economy and social security.
Server management is the key technology of developing high-performance and high availability server. In order to achieve the focused, sustained and effective management of server, the paper designs a hardware platform for high-performance server management.The design uses multi-level architecture to achieve high availability-LMM(local management module) and GMM(global management module). LMM is mainly responsible for the state of the local node, GMM is responsible for monitoring and managing the entire system. The design using a combination of in-band and out-of-band monitoring ways to monitor the compute nodes in real-time. In addition, GMM’s hot backup technology, and multi-channel communication network redundancy ensures system availability. Hot-plug feature is designed to shorter mean maintenance time, so as to improve system availability.
The paper studys the server management technology based on IPMI , the common high-availability technologies, and the needs of the high-performance, high availability server .The paper summarizes the functional requirements of server management platform and then puts forward a double-layer management structure-- LMM and GMM, the detailed design of LMM and GMM is finished at the same time . Finally the hardware of LMM is implemented using a ARM7 minimum system + Peripheral Communication Interface Group , and the main functions of each module is tested.
引文
[1] F. Piedad, M. Hawkins,High Availability: Design, Techniques and Processes, Prentice Hall PTR, 2000.
[2] Haykin S, et al. Adaptive Filtering Theory[M]. 1985.
[3] Macri D. The Scalability Problem. Proceedings of the ACM SIGGRAPH,2004: 66-73.
[4] De Dominicis C M, Ferrar P, Flammini A, et al. Integration of existing IEC61850-based SAS within new high-availability architectures[C]//2010 IEEE International Workshop on Applied Measurements For Power Systems (AMPS):12.
[5]李宁.基于wBEM的分布式服务器管理与诊断系统[D].上海:复旦大学硕士学位论文,2005.
[6]吕晔,曾文华.基于自适应控制理论的自主计算[J].福建电脑, 2009, (4): 101-102.
[7]服务器的可管理性和可利用性. http://www.abc188.com/info/html/wangzhanyunying/jianzhanjingyan/20080417/71880. html.
[8]潘荷新.采用代理服务器管理Internet机房[J].实验室研究与探索, 2001, 20(5).
[9]张磊,李玉龙,胡磊.服务器管理系统双机热备管理软件的设计与实现[J].电脑知识与技术,2009, 15(33): 9419-9421.
[10]吴晓葵. KVM技术在网络服务器管理上的应用[J].西安航空技术高等专科学校学报,2004, 22(1)
[11]高建平,王磊,刘新星等.基于Web的服务器群带外集中监控系统设计与实现[J].计算机与数字工程,2010, 38(8): 70-73.
[12] Bianchini R, Rajamony R, Power and Energy Management for Server Systems[C]. IEEE Computer Society, 2004:68-76.
[13]于治楼,陈乃阔,牛玉峰.基于IPMI的服务器远程管理的研究与实现[J].信息技术与信息化,2010(1).
[14] Intel, Hewlett-Packard, NEC, et al. Intelligent Platform Management Interface Specification Second Generation v2.0[OL].2005.
[15] Yu Z, Ji H. Research of IPMI Management based on BMC SOC[C]. Proc. International Conference on Management and Service Science (MASS), 2010:1-3.
[16] Pridgen A, Julien C. SMASH. Modular Security for Mobile Agents[M]. Berlin: Springer-Verlag, 2007: 99-116.
[17] J. D. Caseetal. Simple network management protocol(snmp). RFC1157, 1990.
[18] Gouvea Da Costa S E, Platts K W, Fleury A, Strategic selection of advanced manufacturing technologies(AMT), based on the manufacturing vision[J]. International Journal of Computer Applications in Technology, 2006, 27(1): 12-23.
[19]谢勇祥,曲道奎. ATCA之机箱管理及高可靠性设计与实现[J].微计算机信息,2007, 23(2-2):42-43.
[20]服务器十大流行技术. http: //server. it168. com/server/2002-04-15/20020405001301. shtml
[21]章琛曦,陈耀武.基于计算机网络的嵌入式系统监控软件远程动态加载方法[N].仪器仪表学报, 2001, 22(4):340-341.
[22]徐家祥.基于分布式环境的实时嵌入式系统监控框架[J].计算机工程, 30(4):158-161.
[23] Li Huimin, Monti A, Ponci F, et al. Voltage Sensor Validation for Decentralized Power System Monitor Using Polynomial Chaos Theory[C]. IEEE Transactions on Instrumentation and Measurement, 2010, 60(5):1633-1643.
[24] PTae W Y, Supervisory control using a new control-relevant switching[J]. Subs of Elsevier Science Inc. 660 White Plains Rd. Tarrytown, NY USA.
[25]彭洲红,陈莉莉,虞致国等.基于IP核的电力系统监控装置SOC设计[J].仪器仪表学报,2004, 25(4):541-542.
[26] Gerald Tesauro, David M Chess, William E Walsh, et al. A MultiAgent System Approach to Autonomic Computing[C]//Proceeding of the International Conference on Autonomous Agents and MultiAgent System. New York: ACM press, 2004:464-471.
[27]林锐,韩永泉.高质量程序设计指南: C++/C语言第3版[M].电子工业出版社, 2007: 22-23
[28]韩振,吴卿,倪永军等.基于自主计算的政务数据中心体系架构的研究[J].计算机应用与软件, 2009, 26(11): 91-94.
[29] Jeffrey O Kephart, David M Chess. The Vision Of Autonomic Computing[J]. IEEE Computer Society,2003:41-50.
[30] Steve R White, James E Hanson, Ian Whalley, et al. An Architectural Approach to Autonomic Computing[C]//Proceedings of the International Conference on Autonomic Computing(ICAC’04). Washington D.C. IEEE Computer Society, 2004: 2 - 9.
[31] Kozak T, Predki P, Makowski D. Real-Time IPMI Protocol Analyzer[C]. Proc. 17th IEEE-NPSS Real Time Conference (RT), 2010:1-7.
[32]娄山林.浅谈IPMI标准[J].科技浪潮, 2007, 54: 25.
[33]孙礼强.基于IPMI技术嵌入式软件自动化测试方案的研究[D].北京:北京邮电大学工程硕士研究生学位论文,2005.
[34]李娜.基于IPMI技术的服务器管理系统的设计与实现[D].北京:北京邮电大学硕士学位论文,2009.
[35] IPMI Specification Second Generation v2.0. www.mtel.com/design/server/ipmi. 2006.6.
[36] Kozak T, Predki P, Makowski D. Real-Time IPMI Protocol Analyzer[C]. Proc. 17th IEEE-NPSS Real Time Conference (RT), 2010:1-7.
[37]李彦,陈德人,梁达明.基于IPMI的虚拟远程串口终端[J].现代机械, 2006, (3): 35-36.
[38]徐拾义可信计算系统设计和分析[M].北京:清华大学出版社, 2006.
[39] Conde R F, Haber J W, Webbert R W. Benefits and lessons learned from the use of the compact PCI standard for spacecraft avionics[C]. Digital Avionics Systems Conference, 2002.
[40] Piotrowski A, Makowski D. PCIExpress Hot-Plug Mechanism in Linux-based ATCA Control Systems[C]// Mixed Design of Integrated Circuits and Systems (MIXDES), Proceedings of the 17th International Conference 24-26 June 2010: 148.
[41] Larsen R S. PICMG xTCA Standards Extensions for Physics: New Developments and Future Plans Real Time Conference (RT), 2010 17th IEEE-NPSS 24-28 May 2010:1.
[42]张亚勇.一种面向事务处理的容错计算机仲裁技术研究与实现[D].哈尔滨:哈尔滨工业大学硕士学位论文,2010.
[43] S3C44B0X Datasheet. Samsung Electronics. 2004.
[44] Sloss A. Arm System Developer’s Guide: Designing and Optimizing System. Elsevier, 2005.
[45]陈阳.嵌入式高可信计算机硬件系统的设计与实现[D].哈尔滨:哈尔滨工业大学硕士学位论文,2005.
[46] Intel StrataFlash? Memory (J3) Samsung Electronics. 2004.
[47] K9F1208U0M Datasheet. Samsung Electronics. 2002.
[48] HY57V561620C(L)T(P) Datasheet. Hynix 2004.
[49] RTL8019AS Datasheet. REALTEK Semiconductor Corp. 2001-5-10.
[50] SL811HST-AC Datasheet.
[51] Linear Technology. CompactPCI Dual Hot Swap Controller LTC1646. 2005.
[52]熊飞.基于ARM7的高可信系统核心模块数据通信的设计与实现[D].哈尔滨:哈尔滨工业大学硕士学位论文,2010.
[53] Brown M. Power Supply Cookbook. Bufferworth-Heinemann. 2001:1-5.
[54] LM1117 Datasheet. National Semiconductor Corporation. 1999.
[55] RTL8019网卡驱动. http: //hyt19860117. love. blog. 163. com/blog/static/127773729201032122152228/
[56] Yi Chen, Yim-Shu Lee. A Hot-Swap Solution for Paralleled Power Modules by Using Current-Sharing Interface Circuits. Power Electronics. 2006(21):1564-1571.
[2] Haykin S, et al. Adaptive Filtering Theory[M]. 1985.
[3] Macri D. The Scalability Problem. Proceedings of the ACM SIGGRAPH,2004: 66-73.
[4] De Dominicis C M, Ferrar P, Flammini A, et al. Integration of existing IEC61850-based SAS within new high-availability architectures[C]//2010 IEEE International Workshop on Applied Measurements For Power Systems (AMPS):12.
[5]李宁.基于wBEM的分布式服务器管理与诊断系统[D].上海:复旦大学硕士学位论文,2005.
[6]吕晔,曾文华.基于自适应控制理论的自主计算[J].福建电脑, 2009, (4): 101-102.
[7]服务器的可管理性和可利用性. http://www.abc188.com/info/html/wangzhanyunying/jianzhanjingyan/20080417/71880. html.
[8]潘荷新.采用代理服务器管理Internet机房[J].实验室研究与探索, 2001, 20(5).
[9]张磊,李玉龙,胡磊.服务器管理系统双机热备管理软件的设计与实现[J].电脑知识与技术,2009, 15(33): 9419-9421.
[10]吴晓葵. KVM技术在网络服务器管理上的应用[J].西安航空技术高等专科学校学报,2004, 22(1)
[11]高建平,王磊,刘新星等.基于Web的服务器群带外集中监控系统设计与实现[J].计算机与数字工程,2010, 38(8): 70-73.
[12] Bianchini R, Rajamony R, Power and Energy Management for Server Systems[C]. IEEE Computer Society, 2004:68-76.
[13]于治楼,陈乃阔,牛玉峰.基于IPMI的服务器远程管理的研究与实现[J].信息技术与信息化,2010(1).
[14] Intel, Hewlett-Packard, NEC, et al. Intelligent Platform Management Interface Specification Second Generation v2.0[OL].2005.
[15] Yu Z, Ji H. Research of IPMI Management based on BMC SOC[C]. Proc. International Conference on Management and Service Science (MASS), 2010:1-3.
[16] Pridgen A, Julien C. SMASH. Modular Security for Mobile Agents[M]. Berlin: Springer-Verlag, 2007: 99-116.
[17] J. D. Caseetal. Simple network management protocol(snmp). RFC1157, 1990.
[18] Gouvea Da Costa S E, Platts K W, Fleury A, Strategic selection of advanced manufacturing technologies(AMT), based on the manufacturing vision[J]. International Journal of Computer Applications in Technology, 2006, 27(1): 12-23.
[19]谢勇祥,曲道奎. ATCA之机箱管理及高可靠性设计与实现[J].微计算机信息,2007, 23(2-2):42-43.
[20]服务器十大流行技术. http: //server. it168. com/server/2002-04-15/20020405001301. shtml
[21]章琛曦,陈耀武.基于计算机网络的嵌入式系统监控软件远程动态加载方法[N].仪器仪表学报, 2001, 22(4):340-341.
[22]徐家祥.基于分布式环境的实时嵌入式系统监控框架[J].计算机工程, 30(4):158-161.
[23] Li Huimin, Monti A, Ponci F, et al. Voltage Sensor Validation for Decentralized Power System Monitor Using Polynomial Chaos Theory[C]. IEEE Transactions on Instrumentation and Measurement, 2010, 60(5):1633-1643.
[24] PTae W Y, Supervisory control using a new control-relevant switching[J]. Subs of Elsevier Science Inc. 660 White Plains Rd. Tarrytown, NY USA.
[25]彭洲红,陈莉莉,虞致国等.基于IP核的电力系统监控装置SOC设计[J].仪器仪表学报,2004, 25(4):541-542.
[26] Gerald Tesauro, David M Chess, William E Walsh, et al. A MultiAgent System Approach to Autonomic Computing[C]//Proceeding of the International Conference on Autonomous Agents and MultiAgent System. New York: ACM press, 2004:464-471.
[27]林锐,韩永泉.高质量程序设计指南: C++/C语言第3版[M].电子工业出版社, 2007: 22-23
[28]韩振,吴卿,倪永军等.基于自主计算的政务数据中心体系架构的研究[J].计算机应用与软件, 2009, 26(11): 91-94.
[29] Jeffrey O Kephart, David M Chess. The Vision Of Autonomic Computing[J]. IEEE Computer Society,2003:41-50.
[30] Steve R White, James E Hanson, Ian Whalley, et al. An Architectural Approach to Autonomic Computing[C]//Proceedings of the International Conference on Autonomic Computing(ICAC’04). Washington D.C. IEEE Computer Society, 2004: 2 - 9.
[31] Kozak T, Predki P, Makowski D. Real-Time IPMI Protocol Analyzer[C]. Proc. 17th IEEE-NPSS Real Time Conference (RT), 2010:1-7.
[32]娄山林.浅谈IPMI标准[J].科技浪潮, 2007, 54: 25.
[33]孙礼强.基于IPMI技术嵌入式软件自动化测试方案的研究[D].北京:北京邮电大学工程硕士研究生学位论文,2005.
[34]李娜.基于IPMI技术的服务器管理系统的设计与实现[D].北京:北京邮电大学硕士学位论文,2009.
[35] IPMI Specification Second Generation v2.0. www.mtel.com/design/server/ipmi. 2006.6.
[36] Kozak T, Predki P, Makowski D. Real-Time IPMI Protocol Analyzer[C]. Proc. 17th IEEE-NPSS Real Time Conference (RT), 2010:1-7.
[37]李彦,陈德人,梁达明.基于IPMI的虚拟远程串口终端[J].现代机械, 2006, (3): 35-36.
[38]徐拾义可信计算系统设计和分析[M].北京:清华大学出版社, 2006.
[39] Conde R F, Haber J W, Webbert R W. Benefits and lessons learned from the use of the compact PCI standard for spacecraft avionics[C]. Digital Avionics Systems Conference, 2002.
[40] Piotrowski A, Makowski D. PCIExpress Hot-Plug Mechanism in Linux-based ATCA Control Systems[C]// Mixed Design of Integrated Circuits and Systems (MIXDES), Proceedings of the 17th International Conference 24-26 June 2010: 148.
[41] Larsen R S. PICMG xTCA Standards Extensions for Physics: New Developments and Future Plans Real Time Conference (RT), 2010 17th IEEE-NPSS 24-28 May 2010:1.
[42]张亚勇.一种面向事务处理的容错计算机仲裁技术研究与实现[D].哈尔滨:哈尔滨工业大学硕士学位论文,2010.
[43] S3C44B0X Datasheet. Samsung Electronics. 2004.
[44] Sloss A. Arm System Developer’s Guide: Designing and Optimizing System. Elsevier, 2005.
[45]陈阳.嵌入式高可信计算机硬件系统的设计与实现[D].哈尔滨:哈尔滨工业大学硕士学位论文,2005.
[46] Intel StrataFlash? Memory (J3) Samsung Electronics. 2004.
[47] K9F1208U0M Datasheet. Samsung Electronics. 2002.
[48] HY57V561620C(L)T(P) Datasheet. Hynix 2004.
[49] RTL8019AS Datasheet. REALTEK Semiconductor Corp. 2001-5-10.
[50] SL811HST-AC Datasheet.
[51] Linear Technology. CompactPCI Dual Hot Swap Controller LTC1646. 2005.
[52]熊飞.基于ARM7的高可信系统核心模块数据通信的设计与实现[D].哈尔滨:哈尔滨工业大学硕士学位论文,2010.
[53] Brown M. Power Supply Cookbook. Bufferworth-Heinemann. 2001:1-5.
[54] LM1117 Datasheet. National Semiconductor Corporation. 1999.
[55] RTL8019网卡驱动. http: //hyt19860117. love. blog. 163. com/blog/static/127773729201032122152228/
[56] Yi Chen, Yim-Shu Lee. A Hot-Swap Solution for Paralleled Power Modules by Using Current-Sharing Interface Circuits. Power Electronics. 2006(21):1564-1571.