开源云计算平台管理支撑平台的研究与实现
|
摘要
近年来,伴随着计算机技术和计算机网络的飞速发展,我们已经进入互联网时代,各种各样的网络应用已经渗透到人们工作与生活的方方面面。互联网为人们生活提供便利的同时,其惊人能耗已经成为国家和社会不得不关注的问题。据调查统计,2011年中国的IT能耗占全国能源耗电总量的5%,而且正在以平均5年增长一倍的速度增长。然而,目前绝大部分服务器的资源利用率并不高,大部分资源都处于空闲状态以应付突如其来的应用高峰。因此,如果能通过提高服务器资源利用率来降低IT能耗,对社会将会有巨大环境效益和经济效益。
云计算技术的出现为提高服务器资源利用率降低能耗提供了有效的途径。云计算技术借助虚拟化技术以管理虚拟基础设施的方式管理物理服务器资源,以资源池的方式按照用户需求提供基础设施资源,具有资源分配方式灵活、分配粒度精确的特点。云计算技术具有灵活和易扩展等优势,弥补了传统运维管理的很多不足,显著增强了管理便捷性、提高了资源利用率和扩展性。
基础设施即服务(IaaS)云计算平台,通过网络把服务器、操作系统、磁盘存储和网络等基础设施资源以服务的形式提供给用户。IaaS云计算平台提供和管理基础设施的云计算平台,作为传统纯物理服务器的运维环境的主要替代模式,被广泛的用作在云端数据中心、平台即服务(PaaS)云计算平台和软件即服务(SaaS)云计算平台的底层设施提供平台。
本文的主要工作是通过研究分析主流的开源IaaS云计算平台,设计并实现适用于多种开源IaaS云计算平台的管理支撑平台,在云计算平台的基础上提供了对云计算平台的监控服务以及基础管理服务,并为服务开发了对外调用接口,通过服务调用接口可以获取云计算平台内物理服务器和虚拟机的负载信息以及便捷的管理云计算平台。基于本文实现的云计算平台的管理支撑平台,用户可以非常方便的实现负载均衡系统或者将云计算平台集成到其他系统中。为了展示本文的云计算平台管理支撑平台,本文以开源的虚拟化软件KVM和开源IaaS云计算平台OpenStack为基础,实现了此管理支撑平台,并基于此平台实现了集自动化调度与门户管理于一体的云计算平台基础运维系统原型。充分的展示了云计算平台管理支撑平台的实际应用意义。
In recent years, as the development of computer hardware, software technology and Internet technology, we have entered the Internet era, and Internet applications have penetrated into all aspects of people's work and life. While Internet provides convenience for people's working and living, the amazing energy consumption has become one of the largest problems to which state and society had to face. According to the survey, in2011, China's IT energy consumption accounted for5%of the total power consumption of the energy, and the growth rate is an average five-year growth doubled. However, the resource utilization of most of servers on which the web applications are running is very low, because most of the computing resource which is used for coping with the sudden application peak is idle for most of the time. Therefore, to reduce energy consumption by improving the utilization of server resources can get huge environmental and economic benefits.
The cloud computing technology provides a new way to solve the IT energy consumption problems. Cloud computing manages physical server resources as virtual resources with the help of virtualization technology, and provide computing resources to users in a form of resource pool, which can set of resources allocation granularity more accurately. Cloud computing technology has many features and advantages to make up for the way of traditional services runs to improve the convenience of management, resource utilization and scalability.
Infrastructure as a Service (IaaS) cloud computing platform provided users servers, operating systems, disk storage, and network resources through the network. IaaS cloud computing platform is a kind of basic cloud computing platform on which data center usually works. Also, IaaS cloud computing platform is can be the basic platform of the Platform as a Service (PaaS) cloud computing platform and Software as a Service (SaaS) cloud computing platform.
The main work of this paper is to analyze the popular open-source IaaS cloud computing platforms, design and implement a management supporting framework based open-source IaaS cloud computing platforms to provide monitoring and objective management services to users. Cloud computing managers and developers can get the workload data of physical and virtual machines by call the interfaces of the monitor service of the management supporting framework, and manage the cloud computing platform in an objective way. Base the framework users can easily develop a load balance system or integrate the cloud computing platform into other systems. In order to demonstrate the management supporting framework better, we implement the framework with a distributed architecture and asynchronous communications based virtualization software KVM, cloud computing platform OpenStack. Besides, we achieved a basic operation and maintenance system with automated scheduling system and portal management interface. The maintenance system fully demonstrated the practical significance of the cloud computing platform management supporting framework.
云计算技术的出现为提高服务器资源利用率降低能耗提供了有效的途径。云计算技术借助虚拟化技术以管理虚拟基础设施的方式管理物理服务器资源,以资源池的方式按照用户需求提供基础设施资源,具有资源分配方式灵活、分配粒度精确的特点。云计算技术具有灵活和易扩展等优势,弥补了传统运维管理的很多不足,显著增强了管理便捷性、提高了资源利用率和扩展性。
基础设施即服务(IaaS)云计算平台,通过网络把服务器、操作系统、磁盘存储和网络等基础设施资源以服务的形式提供给用户。IaaS云计算平台提供和管理基础设施的云计算平台,作为传统纯物理服务器的运维环境的主要替代模式,被广泛的用作在云端数据中心、平台即服务(PaaS)云计算平台和软件即服务(SaaS)云计算平台的底层设施提供平台。
本文的主要工作是通过研究分析主流的开源IaaS云计算平台,设计并实现适用于多种开源IaaS云计算平台的管理支撑平台,在云计算平台的基础上提供了对云计算平台的监控服务以及基础管理服务,并为服务开发了对外调用接口,通过服务调用接口可以获取云计算平台内物理服务器和虚拟机的负载信息以及便捷的管理云计算平台。基于本文实现的云计算平台的管理支撑平台,用户可以非常方便的实现负载均衡系统或者将云计算平台集成到其他系统中。为了展示本文的云计算平台管理支撑平台,本文以开源的虚拟化软件KVM和开源IaaS云计算平台OpenStack为基础,实现了此管理支撑平台,并基于此平台实现了集自动化调度与门户管理于一体的云计算平台基础运维系统原型。充分的展示了云计算平台管理支撑平台的实际应用意义。
In recent years, as the development of computer hardware, software technology and Internet technology, we have entered the Internet era, and Internet applications have penetrated into all aspects of people's work and life. While Internet provides convenience for people's working and living, the amazing energy consumption has become one of the largest problems to which state and society had to face. According to the survey, in2011, China's IT energy consumption accounted for5%of the total power consumption of the energy, and the growth rate is an average five-year growth doubled. However, the resource utilization of most of servers on which the web applications are running is very low, because most of the computing resource which is used for coping with the sudden application peak is idle for most of the time. Therefore, to reduce energy consumption by improving the utilization of server resources can get huge environmental and economic benefits.
The cloud computing technology provides a new way to solve the IT energy consumption problems. Cloud computing manages physical server resources as virtual resources with the help of virtualization technology, and provide computing resources to users in a form of resource pool, which can set of resources allocation granularity more accurately. Cloud computing technology has many features and advantages to make up for the way of traditional services runs to improve the convenience of management, resource utilization and scalability.
Infrastructure as a Service (IaaS) cloud computing platform provided users servers, operating systems, disk storage, and network resources through the network. IaaS cloud computing platform is a kind of basic cloud computing platform on which data center usually works. Also, IaaS cloud computing platform is can be the basic platform of the Platform as a Service (PaaS) cloud computing platform and Software as a Service (SaaS) cloud computing platform.
The main work of this paper is to analyze the popular open-source IaaS cloud computing platforms, design and implement a management supporting framework based open-source IaaS cloud computing platforms to provide monitoring and objective management services to users. Cloud computing managers and developers can get the workload data of physical and virtual machines by call the interfaces of the monitor service of the management supporting framework, and manage the cloud computing platform in an objective way. Base the framework users can easily develop a load balance system or integrate the cloud computing platform into other systems. In order to demonstrate the management supporting framework better, we implement the framework with a distributed architecture and asynchronous communications based virtualization software KVM, cloud computing platform OpenStack. Besides, we achieved a basic operation and maintenance system with automated scheduling system and portal management interface. The maintenance system fully demonstrated the practical significance of the cloud computing platform management supporting framework.
引文
[1]金海.系统虚拟化[M].清华大学出版社,2008.
[2]Hai Jin. Virtualization Technology for Computing System:Opportunities and Challenges[C].The 10th IEEE International Conference on High Performance Computing and Communications,2008, pp3-3.
[3]金海,邓丽,吴松等.一种能耗感知的虚拟集群CPU资源自动再配置方法[J].计算机研究与发展,2011,pp.1123-1133.
[4]陈康,郑伟民.云计算：系统实例与研究现状[J].软件学报,May 2009,PP.1337-1348.
[5]罗军舟,金嘉晖等.云计算：体系架构与关键技术[J]_通信学报,July 2011,32(7),pp.3-21.
[6]叶可江,吴朝晖等.虚拟化云计算平台的能耗管理[J].计算机学报,June 2012,35(6),pp.1262-1285.
[7]Padala P, Shin K G, Zhu X. Adaptive Control of Virtualized Resources in Utility Computing Environments, European Conference on Computer Systems,2007, pp.289-302.
[8]Padala P, Hou K Y, Shin K G. Automated control of multiple virtualized resources[C], Proceedings of the 4th ACM European conference on Computer systems,2009, pp.13-26.
[9]http://www.enet.com.cn/article/2012/0925/A20120925167463.shtml
[10]陈彬.分布环境下虚拟机按需部署关键技术研究[博士论文].国防科技大学,2010.
[11]宋振华.虚拟化技术中的存储管理问题研究[硕士论文],中国科学技术大学,2010.
[12]Sushil Bhardwaj, Leena Jain, Sandeep Jain. Cloud Computing:A Study of Infrastructure as a Service[J].International Journal of Engineering and Information Technology,2010, Vol.2, No.1, pp.60-63.
[13]Armbrust M, Fox A, Griffith R. A view of cloud computing[J],Communications of the ACM,2010,53(4), pp.50-58.
[14]Nurmi D, Wolski R, Grzegorczyk C. The eucalyptus open-source cloud computing system[C]. Cluster Computing and the Grid,2009,9th IEEE/ACM International Symposium on,2009, pp.124-131.
[15]TaheriMonfared, Aryan. As Strong as the Weakest Link:Handling Compromised Components in OpenStack[C]. Cloud Computing Technology and Science,2011 IEEE Third International Conference on,2011, pp.189-196.
[16]Agmon Ben-Yehuda O, Ben-Yehuda M, Schuster A. Deconstructing amazon ec2 spot instance pricing[C]. Cloud Computing Technology and Science,2011 IEEE Third International Conference on,2011, pp.304-311.
[17]Li Song, Gong B, Hu Y. Design and implementation of portlet services of cloud computing platforms[C].Pervasive Computing and Applications (ICPCA),2011 6th International Conference on, IEEE,2011, pp.164-168.
[18]Ando R, Kadobayashi Y, Shinoda Y. An enhancement of trusted domain enforcement using VMM interruption mechanism[C].Young Computer Scientists, 2008,The 9th International Conference for,2008, pp.2222-2229.
[19]徐志红,刘进军,赵生慧.基于同步机制的虚拟机磁盘在线迁移策略[J].计算机工程,2012.
[20]钱磊,李宏亮,谢向辉.虚拟化技术在高性能计算机系统中的应用研究[J].计算机工程与科学,2009.
[21]孟江涛,卢显良.虚拟机监控器Xen的可靠性优化[J].计算机应用,2010.
[22]Libvirt. http://libvirt.org/
[23]高国钊,吕智慧,吴杰.基于Web服务的统一虚拟系统管理[J].计算机工程与设计,2010.
[24]Wang Z, Jiang X. Hypersafe:A lightweight approach to provide lifetime hypervisor control-flow integrity[C].Security and Privacy (SP),2010 IEEE Symposium on,2010, pp.380-395.
[25]Sempolinski P, Thain D. A comparison and critique of eucalyptus, opennebula and nimbus[C].Cloud Computing Technology and Scienc,2010 IEEE Second International Conference on,2010, pp.417-426.
[26]Wen X, Gu G, Li Q. Comparison of open-source cloud management platforms: OpenStack and OpenNebula[C].Fuzzy Systems and Knowledge Discovery (FSKD),20129th International Conference on,2012, pp.2457-2461.
[27]Vinoski S. Advanced message queuing protocol[J].Internet Computing,2006, 10(6),pp.87-89.
[28]Novotny J, Russell M, Wehrens O. GridSphere:an advanced portal framework. Euromicro Conference,2004, Proceedings,30th,2004, pp.412-419.
[29]Martin Brown, http://www.ibm.com/developerworks/cn/grid/gr-tradhp/
[30]陈美荣,蔡洪斌,李武林,等.基于GridSphere的网格门户设计与实现[J].计算机应用研究,2006,23(2),pp.213-215.
[31]陈晓红.面向科学与工程计算应用网格及其关键技术研究与实现[硕士论文].山东大学,2008.
[32]Mink J W. The basal ganglia:focused selection and inhibition of competing motor programs[J].Progress in neurobiology,1996,50(4), pp.381.
[33]郭兵,沈艳,邵子立.绿色计算的重定义与若干探讨[J].计算机学报,2009(012),pp.2311-2319.
[34]Pfister G F. An introduction to the InfiniBand architecture[J].High Performance Mass Storage and Parallel I/O,2001, Vol.42, pp.617-632.
[35]Videla A, Williams J J W. RabbitMQ in action[M]. Manning,2012.
[36]张涛,黄强,毛磊雅,等.一个基于JSON的对象序列化算法[J].计算机工程与应用,2007,43(15)：98-100.
[37]Linux sar man page, http://linux.die.net/man/1/sar
[38]余璐.一种改进QEMU精确异常处理机制的研究[D].华中科技大学,2008.
[39]万国良,武守东VirtualBox安装及使用[J].中国科技信息,2011.
[40]Rosenblum M. VMware's Virtual PlatformTM[C].Proceedings of Hot Chips,1999, pp.185-196.
[41]孔凡敏.开放虚拟化格式及虚拟机在图书馆中的应用[J].科技情报开发与经济,2012.
[2]Hai Jin. Virtualization Technology for Computing System:Opportunities and Challenges[C].The 10th IEEE International Conference on High Performance Computing and Communications,2008, pp3-3.
[3]金海,邓丽,吴松等.一种能耗感知的虚拟集群CPU资源自动再配置方法[J].计算机研究与发展,2011,pp.1123-1133.
[4]陈康,郑伟民.云计算：系统实例与研究现状[J].软件学报,May 2009,PP.1337-1348.
[5]罗军舟,金嘉晖等.云计算：体系架构与关键技术[J]_通信学报,July 2011,32(7),pp.3-21.
[6]叶可江,吴朝晖等.虚拟化云计算平台的能耗管理[J].计算机学报,June 2012,35(6),pp.1262-1285.
[7]Padala P, Shin K G, Zhu X. Adaptive Control of Virtualized Resources in Utility Computing Environments, European Conference on Computer Systems,2007, pp.289-302.
[8]Padala P, Hou K Y, Shin K G. Automated control of multiple virtualized resources[C], Proceedings of the 4th ACM European conference on Computer systems,2009, pp.13-26.
[9]http://www.enet.com.cn/article/2012/0925/A20120925167463.shtml
[10]陈彬.分布环境下虚拟机按需部署关键技术研究[博士论文].国防科技大学,2010.
[11]宋振华.虚拟化技术中的存储管理问题研究[硕士论文],中国科学技术大学,2010.
[12]Sushil Bhardwaj, Leena Jain, Sandeep Jain. Cloud Computing:A Study of Infrastructure as a Service[J].International Journal of Engineering and Information Technology,2010, Vol.2, No.1, pp.60-63.
[13]Armbrust M, Fox A, Griffith R. A view of cloud computing[J],Communications of the ACM,2010,53(4), pp.50-58.
[14]Nurmi D, Wolski R, Grzegorczyk C. The eucalyptus open-source cloud computing system[C]. Cluster Computing and the Grid,2009,9th IEEE/ACM International Symposium on,2009, pp.124-131.
[15]TaheriMonfared, Aryan. As Strong as the Weakest Link:Handling Compromised Components in OpenStack[C]. Cloud Computing Technology and Science,2011 IEEE Third International Conference on,2011, pp.189-196.
[16]Agmon Ben-Yehuda O, Ben-Yehuda M, Schuster A. Deconstructing amazon ec2 spot instance pricing[C]. Cloud Computing Technology and Science,2011 IEEE Third International Conference on,2011, pp.304-311.
[17]Li Song, Gong B, Hu Y. Design and implementation of portlet services of cloud computing platforms[C].Pervasive Computing and Applications (ICPCA),2011 6th International Conference on, IEEE,2011, pp.164-168.
[18]Ando R, Kadobayashi Y, Shinoda Y. An enhancement of trusted domain enforcement using VMM interruption mechanism[C].Young Computer Scientists, 2008,The 9th International Conference for,2008, pp.2222-2229.
[19]徐志红,刘进军,赵生慧.基于同步机制的虚拟机磁盘在线迁移策略[J].计算机工程,2012.
[20]钱磊,李宏亮,谢向辉.虚拟化技术在高性能计算机系统中的应用研究[J].计算机工程与科学,2009.
[21]孟江涛,卢显良.虚拟机监控器Xen的可靠性优化[J].计算机应用,2010.
[22]Libvirt. http://libvirt.org/
[23]高国钊,吕智慧,吴杰.基于Web服务的统一虚拟系统管理[J].计算机工程与设计,2010.
[24]Wang Z, Jiang X. Hypersafe:A lightweight approach to provide lifetime hypervisor control-flow integrity[C].Security and Privacy (SP),2010 IEEE Symposium on,2010, pp.380-395.
[25]Sempolinski P, Thain D. A comparison and critique of eucalyptus, opennebula and nimbus[C].Cloud Computing Technology and Scienc,2010 IEEE Second International Conference on,2010, pp.417-426.
[26]Wen X, Gu G, Li Q. Comparison of open-source cloud management platforms: OpenStack and OpenNebula[C].Fuzzy Systems and Knowledge Discovery (FSKD),20129th International Conference on,2012, pp.2457-2461.
[27]Vinoski S. Advanced message queuing protocol[J].Internet Computing,2006, 10(6),pp.87-89.
[28]Novotny J, Russell M, Wehrens O. GridSphere:an advanced portal framework. Euromicro Conference,2004, Proceedings,30th,2004, pp.412-419.
[29]Martin Brown, http://www.ibm.com/developerworks/cn/grid/gr-tradhp/
[30]陈美荣,蔡洪斌,李武林,等.基于GridSphere的网格门户设计与实现[J].计算机应用研究,2006,23(2),pp.213-215.
[31]陈晓红.面向科学与工程计算应用网格及其关键技术研究与实现[硕士论文].山东大学,2008.
[32]Mink J W. The basal ganglia:focused selection and inhibition of competing motor programs[J].Progress in neurobiology,1996,50(4), pp.381.
[33]郭兵,沈艳,邵子立.绿色计算的重定义与若干探讨[J].计算机学报,2009(012),pp.2311-2319.
[34]Pfister G F. An introduction to the InfiniBand architecture[J].High Performance Mass Storage and Parallel I/O,2001, Vol.42, pp.617-632.
[35]Videla A, Williams J J W. RabbitMQ in action[M]. Manning,2012.
[36]张涛,黄强,毛磊雅,等.一个基于JSON的对象序列化算法[J].计算机工程与应用,2007,43(15)：98-100.
[37]Linux sar man page, http://linux.die.net/man/1/sar
[38]余璐.一种改进QEMU精确异常处理机制的研究[D].华中科技大学,2008.
[39]万国良,武守东VirtualBox安装及使用[J].中国科技信息,2011.
[40]Rosenblum M. VMware's Virtual PlatformTM[C].Proceedings of Hot Chips,1999, pp.185-196.
[41]孔凡敏.开放虚拟化格式及虚拟机在图书馆中的应用[J].科技情报开发与经济,2012.