虚拟化环境中多目标负载均衡的研究
摘要
近年来,虚拟化技术以其良好的独立性、高的资源利用率及易管理性而备受关注。目前常用的虚拟化软件Xen具有占用资源少、性能高和开源等优点,常被用来搭建高性能和高可靠的虚拟化环境。在基于Xen的虚拟化环境中,物理机超载将引起物理机负载失衡,严重影响其上运行的虚拟机性能甚至导致宕机。因此,如何实现虚拟化环境中物理机的负载均衡成为研究的一个热点。
论文描述了虚拟化技术的发展历史、研究背景、国内外研究现状和研究意义,分析了虚拟机系统的基本原理和实现的关键技术,重点阐述了Xen的虚拟化技术、实时迁移技术、Libvirt技术。借鉴国内外物理机负载均衡研究相关文献和思路,综合考虑虚拟机的迁移开销、物理机的CPU和内存资源的负载平衡,提出了一种多目标负载均衡方法。该方法首先利用时间序列模型预测超载物理机,其次利用最小化迁移开销算法为超载物理机选择待迁虚拟机,然后采用降序最佳适应算法为待迁虚拟机选择目标物理机,最后利用Xen的实时迁移命令触发迁移。
为了对多目标负载均衡方法的性能进行验证,在实际的基于Xen的教学虚拟化系统中,应用了多目标负载均衡方法管理系统中集群,应用结果表明,多目标负载均衡方法能在很短的时间内以较小的迁移开销实现物理服务器的CPU和内存资源的负载均衡,有效改善了虚拟化集群的性能。
The virtualization technology is concerned because of its independence and high resource utilization and easy management in recent years. The common virtualization software Xen that has the advantages of the less resource-intensive, high-performance and open source is often used to build the high-performance and high reliability virtualization environment. The overload of the physical machines will seriously affect the performance of the running virtual machine and even have them go down int the virtualization environment based on the Xen, therefore, how to achieve load balancing of the physical machines in the virtualized environment has become a hot research topic.
The paper described the history of the development of virtualization technology, the research background, the research status and the significance, and analysised the basic principle of the virtual machine system and the key technologies of the achievement, focused on the Xen virtualization, live migration technology, Libvirt. This paper proposed a multi-goal load balancing method drawing on ideas and methods of the related literature taking into account three factors of the migration overhead, load balancing of the physical machine's CPU and memory resources. First, it uses the time series model to predict the overloaded physical machine, second, it uses the minimize migration overhead algorithm to select the virtual machine to be moved for the overloaded physical machine, third, it uses the descending best-fit algorithm to select the target physical machine for the virtual machine to be moved, finally, it uses the live migration command of the Xen to trigger the migration.
To validate the performance of the multi-goal load balancing method, it is used to manage the cluster of the real teaching virtualization system based on the Xen. The experimental results show the method can achieve the load balancing of CPU and memory of the physical machines of the cluster with a lower migration overhead in a very short period of time and effectively improve the performance of the cluster.
论文描述了虚拟化技术的发展历史、研究背景、国内外研究现状和研究意义,分析了虚拟机系统的基本原理和实现的关键技术,重点阐述了Xen的虚拟化技术、实时迁移技术、Libvirt技术。借鉴国内外物理机负载均衡研究相关文献和思路,综合考虑虚拟机的迁移开销、物理机的CPU和内存资源的负载平衡,提出了一种多目标负载均衡方法。该方法首先利用时间序列模型预测超载物理机,其次利用最小化迁移开销算法为超载物理机选择待迁虚拟机,然后采用降序最佳适应算法为待迁虚拟机选择目标物理机,最后利用Xen的实时迁移命令触发迁移。
为了对多目标负载均衡方法的性能进行验证,在实际的基于Xen的教学虚拟化系统中,应用了多目标负载均衡方法管理系统中集群,应用结果表明,多目标负载均衡方法能在很短的时间内以较小的迁移开销实现物理服务器的CPU和内存资源的负载均衡,有效改善了虚拟化集群的性能。
The virtualization technology is concerned because of its independence and high resource utilization and easy management in recent years. The common virtualization software Xen that has the advantages of the less resource-intensive, high-performance and open source is often used to build the high-performance and high reliability virtualization environment. The overload of the physical machines will seriously affect the performance of the running virtual machine and even have them go down int the virtualization environment based on the Xen, therefore, how to achieve load balancing of the physical machines in the virtualized environment has become a hot research topic.
The paper described the history of the development of virtualization technology, the research background, the research status and the significance, and analysised the basic principle of the virtual machine system and the key technologies of the achievement, focused on the Xen virtualization, live migration technology, Libvirt. This paper proposed a multi-goal load balancing method drawing on ideas and methods of the related literature taking into account three factors of the migration overhead, load balancing of the physical machine's CPU and memory resources. First, it uses the time series model to predict the overloaded physical machine, second, it uses the minimize migration overhead algorithm to select the virtual machine to be moved for the overloaded physical machine, third, it uses the descending best-fit algorithm to select the target physical machine for the virtual machine to be moved, finally, it uses the live migration command of the Xen to trigger the migration.
To validate the performance of the multi-goal load balancing method, it is used to manage the cluster of the real teaching virtualization system based on the Xen. The experimental results show the method can achieve the load balancing of CPU and memory of the physical machines of the cluster with a lower migration overhead in a very short period of time and effectively improve the performance of the cluster.
引文
[1]Jeff, D. Server virtualization architecture and implementation [J]. Crossroads,2009,16(1): 8-12
[2]Haikun Liu., Hai Jin, Xiaofei Liao, etc. Live migration of virtual machine based on full system trace and replay [C]. In:Proceedings of the 18th ACM international symposium on High performance distributed computing. Garching, Germany:ACM,2009.101-109
[3]Seawright L.H., MacKinnon R.A. VM/370-A study of multiplicity and usefulness [J]. IBM System Journal.1979,18(1):4-17
[4]Uhlig R., Neiger G., Rodgers D, etc. Intel virtualization technology [J]. Computer,2005,38(5): 48-56
[5]Yunfa Li, Wanqing Li, Congfeng Jiang. A survey of virtual machine system: currenttechnology and future trends[C]. In:2010 Third International Symposium on Electronic Commerce and Security. Guangzhou, China:IEEE CPS Press,2010.332-336
[6]AMD. AMD I/O Vitualization Technology (IOMMU) Specification[EB].2007. http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/34434.pdf
[7]张建勋,古志民,郑超.云计算研究进展综述[J].计算机应用研究,2010,27(2)：429～433
[8]Boss G, Malladi P, Quan D, etc. Cloud computing[EB].2007. http://download.boulder.ibm.com/ibmdl/pub/software/dw/wes/hipods/Cloud_computing_wp_f inal_8Oct.pdf
[9]金海等.计算系统虚拟化--原理与应用[M].北京：清华大学出版社,2008.1-379
[10]VMware. VMware virtual machine technology [EB].2012. http://www.vmware.com
[11]Microsoft. Microsoft virtual PC [EB].2012. http://www.microsoft.com/windows/virtual-pc/
[12]Xen. The Xen project[EB].2012. http://www.cl.cam.ac.uk/Research/SRG/netos/xen/
[13]Paul Barham, Boris Dragovic, Keir Fraser, etc. Xen and the art of virtualization[C]. In: Proceedings of The 19th ACM Symposium on Operating Systems Principles. New York, USA:ACM Press,2003.164-177
[14]Chris Takemura, Luke S. Crawford. The book of Xen [M]. San Francisco:No Starch Press, 2010.1-251
[15]石磊,邹德清,金海.Xen虚拟化技术[M].武汉：华中科技大学出版社,2009.1-418
[16]VMware. Resource Management with VMware DRS[EB].2006. http://www.vmware.com/pdf/vmware_drs_wp.pdf
[17]Timothy Wood, Prashant Shenoy, Arun Venkataramani,etc. Black-box and Gray-box Strategies for Virtual Machine Migration[C]. In:Proceedings of the 4th Int'l Conference on Networked Systems Design& Implementation. USA:IEEE Press,2007.229-242
[18]Robert Bradford, Evangelos Kotsovinos, Anja Feldmann, etc. Live wide-area migration of virtual machines including local persistent state [C]. In:Proceedings of the 3rd international conference on Virtual execution environments. San Diego, Calornia, USA:ACM, 2007.169-179
[19]Tracy Kimbrel, Baruch Schieber, Maxim Sviridenko. Minimizing migrations in fair multiprocessor scheduling of persistent tasks [C]. In:Proceedings of the fteenth annual ACM-SIAM symposium on Discrete algorithms. New Orleans, Louisiana:Society for Industrial and Applied Mathematics,2004,982-991
[20]肖臻,宋维佳,陈琪等.一种虚拟机调度方法[P].中国专利,200910238364.2010-06-02
[21]刘媛媛,高庆一,陈阳.虚拟计算环境下虚拟机资源负载均衡方法[J].计算机工程,2010,36(16)：30-32
[22]Zhou Wen-yu, Yang Shou-bao, Fang Jun, etc. VMCTune:A Load Balancing Scheme for Virtual Machine Cluster Using Dynamic Resource Allocation[C]. In:2010 Ninth International Conference on Grid and Cloud Computing. USA:IEEE Press,2010.81-86
[23]陈建海,何钦铭,李星等.虚拟化集群负载在多台物理机中均衡分配的方法[P].中国专利,201110199822.3.2011-11-09
[24]颜燕娜.虚拟机平台下虚拟资源的自适应分配策略与机制研究[D].[硕士学位论文].杭州：杭州电子科技大学,2009
[25]刘进军,陈桂林,胡成祥.基于负载特征的虚拟机迁移调度策略[J].计算机工程,2011,37(17)：276-278
[26]Renato J.Figueiredo, Peter A.Dinda, Jose A.B.Fortes. A Case For Grid Computing On Virtual Machines[C]. In:ICDCS'03 Proceedings of the 23rd International Conference on Distributed Computing Systems. Conference Publications,2003,550-559
[27]Ming Zhao, Jian Zhang, Renato Figueiredo. Distributed File System Support for Virtual Machines in Grid Computing[C]. In:HPDC'04:Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing. Conference Publications,2004, 202-211
[28]Neiger,G., Santoni,A., Leung,F., etc. Intel(?) Virtualization Technology:Hardware Support for Efficient Proccessor Virtualization[J]. Intel(?) Technology Journal,2006,10(3):167-178
[29]Qumranet Inc. KVM:Kernel-based Virtualization Driver White Paper[[EB]].2006. Qumranet
[30]A.Whitaker, R.S.Cox, M.Shaw, etc. Constructing Services with Interposable Virtual Hardware[C]. In:Proceedings of the First Symposium on Networked Systems Design and Implementation (NSDI'04). San Francisco, USA:USENIX Press.2004.
[31]K.Admas, O.Agesen. A comparison of software and hardware techniques for x86 virtualization[C]. In:Proceedings of the 12th international conference on Architectural support for programming languages and operating systems. New York, USA,2006.2-13
[32]J.L.Vasseur, V.Uhlig, B.Leslie, etc. Pre-Virtualization:Uniting Two Worlds[C]. In: Proceedings of the twentieth ACM symposium on Operating systems principles. New York, USA,2005:1-2
[33]孙昱.虚拟机Xen及其实时迁移技术研究[D].[硕士学位论文].上海：上海交通大学,2008
[34]Haikun Liu, Hai Jin, Xiaofei Liao, etc. Live virtual machine migration via asynchronous replication and state synchronization [J]. Transactions on Parallel and Distributed Systems, 2011,22(12):1986-1999
[35]江雪.基于Xen虚拟机的动态迁移技术研究[D].[硕士学位论文].成都：电子科技大学,2009
[36]Christopher Clark, Keir Fraser, Steven Hand, etc. Live Migration of Virtual Machines[C]. In: Proceedings of the 2nd Int'l Conference on Networked Systems Design& Implementation. Boston:ACM Press,2005.273-286
[37]KhaledZ.Ibrahim, Steven Hofmeyr, CostinIancu, etc. Optimized pre-copy live migration for memory intensive applications [C]. In:Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, New York, USA:IEEE Press.2011.1-11
[38]怀进鹏,李沁,胡春明.基于虚拟机的虚拟计算环境研究与设计[J].软件学报,2007,18(8)：2016-2026.
[39]http://libvirt.org/
[40]http://libvirt.org/downloads.html
[41]王黎明,王连.应用时间序列分析[M].上海：复旦大学出版社,2009.1-287
[42]Box G P, Jenkins G M, Reinsel G C. Time Series Analysis:Forecasting and Control[M]. USA:Prentice-Hall,1994.1-598
[2]Haikun Liu., Hai Jin, Xiaofei Liao, etc. Live migration of virtual machine based on full system trace and replay [C]. In:Proceedings of the 18th ACM international symposium on High performance distributed computing. Garching, Germany:ACM,2009.101-109
[3]Seawright L.H., MacKinnon R.A. VM/370-A study of multiplicity and usefulness [J]. IBM System Journal.1979,18(1):4-17
[4]Uhlig R., Neiger G., Rodgers D, etc. Intel virtualization technology [J]. Computer,2005,38(5): 48-56
[5]Yunfa Li, Wanqing Li, Congfeng Jiang. A survey of virtual machine system: currenttechnology and future trends[C]. In:2010 Third International Symposium on Electronic Commerce and Security. Guangzhou, China:IEEE CPS Press,2010.332-336
[6]AMD. AMD I/O Vitualization Technology (IOMMU) Specification[EB].2007. http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/34434.pdf
[7]张建勋,古志民,郑超.云计算研究进展综述[J].计算机应用研究,2010,27(2)：429～433
[8]Boss G, Malladi P, Quan D, etc. Cloud computing[EB].2007. http://download.boulder.ibm.com/ibmdl/pub/software/dw/wes/hipods/Cloud_computing_wp_f inal_8Oct.pdf
[9]金海等.计算系统虚拟化--原理与应用[M].北京：清华大学出版社,2008.1-379
[10]VMware. VMware virtual machine technology [EB].2012. http://www.vmware.com
[11]Microsoft. Microsoft virtual PC [EB].2012. http://www.microsoft.com/windows/virtual-pc/
[12]Xen. The Xen project[EB].2012. http://www.cl.cam.ac.uk/Research/SRG/netos/xen/
[13]Paul Barham, Boris Dragovic, Keir Fraser, etc. Xen and the art of virtualization[C]. In: Proceedings of The 19th ACM Symposium on Operating Systems Principles. New York, USA:ACM Press,2003.164-177
[14]Chris Takemura, Luke S. Crawford. The book of Xen [M]. San Francisco:No Starch Press, 2010.1-251
[15]石磊,邹德清,金海.Xen虚拟化技术[M].武汉：华中科技大学出版社,2009.1-418
[16]VMware. Resource Management with VMware DRS[EB].2006. http://www.vmware.com/pdf/vmware_drs_wp.pdf
[17]Timothy Wood, Prashant Shenoy, Arun Venkataramani,etc. Black-box and Gray-box Strategies for Virtual Machine Migration[C]. In:Proceedings of the 4th Int'l Conference on Networked Systems Design& Implementation. USA:IEEE Press,2007.229-242
[18]Robert Bradford, Evangelos Kotsovinos, Anja Feldmann, etc. Live wide-area migration of virtual machines including local persistent state [C]. In:Proceedings of the 3rd international conference on Virtual execution environments. San Diego, Calornia, USA:ACM, 2007.169-179
[19]Tracy Kimbrel, Baruch Schieber, Maxim Sviridenko. Minimizing migrations in fair multiprocessor scheduling of persistent tasks [C]. In:Proceedings of the fteenth annual ACM-SIAM symposium on Discrete algorithms. New Orleans, Louisiana:Society for Industrial and Applied Mathematics,2004,982-991
[20]肖臻,宋维佳,陈琪等.一种虚拟机调度方法[P].中国专利,200910238364.2010-06-02
[21]刘媛媛,高庆一,陈阳.虚拟计算环境下虚拟机资源负载均衡方法[J].计算机工程,2010,36(16)：30-32
[22]Zhou Wen-yu, Yang Shou-bao, Fang Jun, etc. VMCTune:A Load Balancing Scheme for Virtual Machine Cluster Using Dynamic Resource Allocation[C]. In:2010 Ninth International Conference on Grid and Cloud Computing. USA:IEEE Press,2010.81-86
[23]陈建海,何钦铭,李星等.虚拟化集群负载在多台物理机中均衡分配的方法[P].中国专利,201110199822.3.2011-11-09
[24]颜燕娜.虚拟机平台下虚拟资源的自适应分配策略与机制研究[D].[硕士学位论文].杭州：杭州电子科技大学,2009
[25]刘进军,陈桂林,胡成祥.基于负载特征的虚拟机迁移调度策略[J].计算机工程,2011,37(17)：276-278
[26]Renato J.Figueiredo, Peter A.Dinda, Jose A.B.Fortes. A Case For Grid Computing On Virtual Machines[C]. In:ICDCS'03 Proceedings of the 23rd International Conference on Distributed Computing Systems. Conference Publications,2003,550-559
[27]Ming Zhao, Jian Zhang, Renato Figueiredo. Distributed File System Support for Virtual Machines in Grid Computing[C]. In:HPDC'04:Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing. Conference Publications,2004, 202-211
[28]Neiger,G., Santoni,A., Leung,F., etc. Intel(?) Virtualization Technology:Hardware Support for Efficient Proccessor Virtualization[J]. Intel(?) Technology Journal,2006,10(3):167-178
[29]Qumranet Inc. KVM:Kernel-based Virtualization Driver White Paper[[EB]].2006. Qumranet
[30]A.Whitaker, R.S.Cox, M.Shaw, etc. Constructing Services with Interposable Virtual Hardware[C]. In:Proceedings of the First Symposium on Networked Systems Design and Implementation (NSDI'04). San Francisco, USA:USENIX Press.2004.
[31]K.Admas, O.Agesen. A comparison of software and hardware techniques for x86 virtualization[C]. In:Proceedings of the 12th international conference on Architectural support for programming languages and operating systems. New York, USA,2006.2-13
[32]J.L.Vasseur, V.Uhlig, B.Leslie, etc. Pre-Virtualization:Uniting Two Worlds[C]. In: Proceedings of the twentieth ACM symposium on Operating systems principles. New York, USA,2005:1-2
[33]孙昱.虚拟机Xen及其实时迁移技术研究[D].[硕士学位论文].上海：上海交通大学,2008
[34]Haikun Liu, Hai Jin, Xiaofei Liao, etc. Live virtual machine migration via asynchronous replication and state synchronization [J]. Transactions on Parallel and Distributed Systems, 2011,22(12):1986-1999
[35]江雪.基于Xen虚拟机的动态迁移技术研究[D].[硕士学位论文].成都：电子科技大学,2009
[36]Christopher Clark, Keir Fraser, Steven Hand, etc. Live Migration of Virtual Machines[C]. In: Proceedings of the 2nd Int'l Conference on Networked Systems Design& Implementation. Boston:ACM Press,2005.273-286
[37]KhaledZ.Ibrahim, Steven Hofmeyr, CostinIancu, etc. Optimized pre-copy live migration for memory intensive applications [C]. In:Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, New York, USA:IEEE Press.2011.1-11
[38]怀进鹏,李沁,胡春明.基于虚拟机的虚拟计算环境研究与设计[J].软件学报,2007,18(8)：2016-2026.
[39]http://libvirt.org/
[40]http://libvirt.org/downloads.html
[41]王黎明,王连.应用时间序列分析[M].上海：复旦大学出版社,2009.1-287
[42]Box G P, Jenkins G M, Reinsel G C. Time Series Analysis:Forecasting and Control[M]. USA:Prentice-Hall,1994.1-598