VCPU组调度技术的研究与实现
|
摘要
近年来,随着在越来越多的领域中得到广泛应用,虚拟化技术得到了迅速发展。与此同时,由于不同应用场景对虚拟化技术有着不同的功能、性能要求,又给虚拟化技术带来许多新的问题。
当前,虚拟机调度器在调度和CPU资源分配中主要考虑到各个虚拟机间的独立性,而忽略了虚拟机间的一些联系。这种情况下,调度器降低了有协同需求的虚拟机间并行工作的可能性,从而影响了需要多个任务协同工作的大型应用的服务质量;在商业服务器虚拟化中,客户期望最大限度利用自己已付费购买的计算资源,但在传统的虚拟机调度模式中,客户的虚拟机负载不均时,负载较重虚拟机也不能获得该客户其它空闲虚拟机的资源,即:客户付费范围内有些虚拟机资源是闲置的而有些又是负载过重的。
针对上述虚拟机协同问题,本文提出了一种虚拟化平台上的协同调度策略。文章首先分析了虚拟机间的协同关系,根据应用对多虚拟机并行运行的严格性要求不同,将协同请求划分为强协同和弱协同两个等级,然后在此基础上讨论了协同调度中VCPU(Virtual CPU)分配问题和VCPU同步问题。为防止协同VPCU运行在同一物理节点上,本文提出了VCPU分配机制;在VCPU同步上,为了降低协同对系统性能影响,文章综合分析了VCPU抢占条件和时机,同时借鉴了co-scheduling调度算法,提出了基于协同请求级别和VCPU运行优先级的抢占机制;最后结合Xen credit调度算法提出了co-credit协同调度策略。
针对客户虚拟机资源控制问题,为避免出现客户付费范围内资源浪费情况,本文提出了以虚拟机组为单位的计算资源管理方式,同组的虚拟机间可动态共享CPU资源。首先结合多虚拟机资源管理的特点分析了资源分配策略,选取合适的CPU资源分配机制;为了有效管理空闲虚拟机资源,提出了虚拟机状态检测机制,根据虚拟机调度器和虚拟机组不同需求,设计了不同的检测标准;最后根据组内虚拟机状态和VCPU数量等信息,实现了组内空闲CPU资源重新分配。
基于以上分析研究,本文结合开源项目Xen设计开发实现了虚拟机组调度器,并对其性能进行了测试。通过实验表明,协同调度明显提高了虚拟机间协同工作能力,且没有对调度的公平性造成明显影响;虚拟机组资源管理方式使组内繁忙虚拟机能得到组内空闲资源,实现组内资源的有效利用。本课题得到了863项目“可扩展高性能虚拟机关键技术研究”的资助。
Virtualization technology has been developing rapidly in recent years, and is used in more and more areas. Different applications have different requirements of virtualization functions and performance, which brings new problems in virtualization development.
Currently, about CPU resources allocation and scheduling, the Virtual Machine(VM) scheduler mainly cares about independence among VMs, ignoring cooperations among VMs. In this case, the VM scheduler reduces the parallelizability of VMs, which affects the QOS of large-scale applications that need multiple VMs to work cooperatively. In commercial server virtualization environment, customers expect to maximize the utilization of computing resources that they have paid for. However, in the traditional VM scheduling, heavy load VMs can not get resources of the other idle VMs, when the customer’s VMs load imbalance. As a result, some of VMs’resources paid by customer are idle while the others are overloaded.
To improve the ability of VM cooperation, a co-scheduling strategy is proposed for the virtualization environment. First, the VM cooperation is analysed. Cooperations are classified into strong cooperation and weak cooperation according to different requirements of synchronization. Based on the above analysis, VCPUs(Virtual CPU) distribution and synchronization are discussed. In order to prevent cooperative VCPUs from running on the same physical core, a VCPU distribution mechanism is proposed. About VCPU synchronization, for minimizing the impact of cooperation on system performance, conditions and timing of VCPU preemption are analysed comprehensively, then a preemption mechanism is proposed based on traditional parallel system co-scheduling. Finally, co-credit co-scheduling is developed based on Xen credit scheduling.
To manage the curomers’virtual machine resources more efficiently, virtual machine group is treated as the resource allocation unit, so that the same group VMs can share CPU resources dynamicly, and avoid wastage of resources paid by customer. First, Resources allocation strategy is discussed according to multiple VMs resources management features and an appropriate CPU resource allocation mechanism is chosen. To effectively manage the resources of idle VMs, VM state detection mechanism is proposed and different test rules are designed according to the different needs of VM scheduler and VM group. The group idle CPU resources are re-allocated according to the state of VM within the group and the number of VCPUs, which achieves efficient use of group idle resources .
Based on the above analysis, a VM co-scheduler is designed and implemented in the open source Xen project. Finally, experiments show that VM co-scheduler improves the ability of VMs to cooperate and has few effects on the fairness. The loaded VMs within a group can get CPU resources of the idle ones, so that resources within a group are fully utilized. This thesis is supported by 863 project“Scalable high-performance virtual machine key technology research”.
当前,虚拟机调度器在调度和CPU资源分配中主要考虑到各个虚拟机间的独立性,而忽略了虚拟机间的一些联系。这种情况下,调度器降低了有协同需求的虚拟机间并行工作的可能性,从而影响了需要多个任务协同工作的大型应用的服务质量;在商业服务器虚拟化中,客户期望最大限度利用自己已付费购买的计算资源,但在传统的虚拟机调度模式中,客户的虚拟机负载不均时,负载较重虚拟机也不能获得该客户其它空闲虚拟机的资源,即:客户付费范围内有些虚拟机资源是闲置的而有些又是负载过重的。
针对上述虚拟机协同问题,本文提出了一种虚拟化平台上的协同调度策略。文章首先分析了虚拟机间的协同关系,根据应用对多虚拟机并行运行的严格性要求不同,将协同请求划分为强协同和弱协同两个等级,然后在此基础上讨论了协同调度中VCPU(Virtual CPU)分配问题和VCPU同步问题。为防止协同VPCU运行在同一物理节点上,本文提出了VCPU分配机制;在VCPU同步上,为了降低协同对系统性能影响,文章综合分析了VCPU抢占条件和时机,同时借鉴了co-scheduling调度算法,提出了基于协同请求级别和VCPU运行优先级的抢占机制;最后结合Xen credit调度算法提出了co-credit协同调度策略。
针对客户虚拟机资源控制问题,为避免出现客户付费范围内资源浪费情况,本文提出了以虚拟机组为单位的计算资源管理方式,同组的虚拟机间可动态共享CPU资源。首先结合多虚拟机资源管理的特点分析了资源分配策略,选取合适的CPU资源分配机制;为了有效管理空闲虚拟机资源,提出了虚拟机状态检测机制,根据虚拟机调度器和虚拟机组不同需求,设计了不同的检测标准;最后根据组内虚拟机状态和VCPU数量等信息,实现了组内空闲CPU资源重新分配。
基于以上分析研究,本文结合开源项目Xen设计开发实现了虚拟机组调度器,并对其性能进行了测试。通过实验表明,协同调度明显提高了虚拟机间协同工作能力,且没有对调度的公平性造成明显影响;虚拟机组资源管理方式使组内繁忙虚拟机能得到组内空闲资源,实现组内资源的有效利用。本课题得到了863项目“可扩展高性能虚拟机关键技术研究”的资助。
Virtualization technology has been developing rapidly in recent years, and is used in more and more areas. Different applications have different requirements of virtualization functions and performance, which brings new problems in virtualization development.
Currently, about CPU resources allocation and scheduling, the Virtual Machine(VM) scheduler mainly cares about independence among VMs, ignoring cooperations among VMs. In this case, the VM scheduler reduces the parallelizability of VMs, which affects the QOS of large-scale applications that need multiple VMs to work cooperatively. In commercial server virtualization environment, customers expect to maximize the utilization of computing resources that they have paid for. However, in the traditional VM scheduling, heavy load VMs can not get resources of the other idle VMs, when the customer’s VMs load imbalance. As a result, some of VMs’resources paid by customer are idle while the others are overloaded.
To improve the ability of VM cooperation, a co-scheduling strategy is proposed for the virtualization environment. First, the VM cooperation is analysed. Cooperations are classified into strong cooperation and weak cooperation according to different requirements of synchronization. Based on the above analysis, VCPUs(Virtual CPU) distribution and synchronization are discussed. In order to prevent cooperative VCPUs from running on the same physical core, a VCPU distribution mechanism is proposed. About VCPU synchronization, for minimizing the impact of cooperation on system performance, conditions and timing of VCPU preemption are analysed comprehensively, then a preemption mechanism is proposed based on traditional parallel system co-scheduling. Finally, co-credit co-scheduling is developed based on Xen credit scheduling.
To manage the curomers’virtual machine resources more efficiently, virtual machine group is treated as the resource allocation unit, so that the same group VMs can share CPU resources dynamicly, and avoid wastage of resources paid by customer. First, Resources allocation strategy is discussed according to multiple VMs resources management features and an appropriate CPU resource allocation mechanism is chosen. To effectively manage the resources of idle VMs, VM state detection mechanism is proposed and different test rules are designed according to the different needs of VM scheduler and VM group. The group idle CPU resources are re-allocated according to the state of VM within the group and the number of VCPUs, which achieves efficient use of group idle resources .
Based on the above analysis, a VM co-scheduler is designed and implemented in the open source Xen project. Finally, experiments show that VM co-scheduler improves the ability of VMs to cooperate and has few effects on the fairness. The loaded VMs within a group can get CPU resources of the idle ones, so that resources within a group are fully utilized. This thesis is supported by 863 project“Scalable high-performance virtual machine key technology research”.
引文
[1] Woodbury, P., Wilson A. et al. Shared Memory Multiprocessors:The Right Approach to Parallel Processing. CH2686-4/89/0000/0072 1989 IEEE
[2] Gine. F, Solsona. F, Hanzich. M, Hernandez. P and Luque. E. Cooperating CoScheduling: A coscheduling proposal aimed at Non-Dedicated Heterogeneous NOWs. JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY 22(5):695～710 Sept.2007
[3] Sriram Govindan, Arjun R. Nath, Amitayu Das, Bhuvan Urgaonkar, Anand Sivasubramaniam. Xen and Co.: Communication-aware CPU Scheduling for Consolidated Xen-based Hosting Platforms. VEE’07, June 13–15, 2007, San Diego, California, USA.
[4] Dror G. Feitelson, Larry Rudolph. Coscheduling Based on Runtime Identification of Activity Working Sets. International Journal of Parallel Programming, Vol. 23, No. 2, 1995
[5] Alexandru Romosan, Doron Rotem, Arie Shoshani and Derek Wright. Co-Scheduling of Computation and Data on Computer Clusters. Proceedings of the 17th international conference on Scientific and statistical database management table of contents.Santa Barbara, CA 2005 ISBN:1-88888-111-X
[6] Dror G. Feitelson, Larry Rudolph. Gang Scheduling Performance Benefits for Fine-Grain Synchronization. Journal of parallel and distributed computing ISSN 0743-7315 1992
[7] Yair Wiseman and Dror G.Feitelson. Paired Gang Scheduling. IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, VOL. 14, NO. 6, JUNE 2003
[8] Dror G. Feitelson, Larry Rudolph. Parallel Job Scheduling—A Status Report. 10th Workshop on Job Scheduling Strategies for Parallel processing Held in conjunction with SIGMETRICS 2004, New York, NY, June 13, 2004
[9] Diego Ongaro, Alan L. Cox, Scott Rixner. Scheduling I/O in Virtual Machine Monitors. VEE’08, March 5–7, 2008, Seattle, Washington, USA
[10] Zafeirios C. Papazachos, Helen D. Karatza. The impact of task service time variability on gang scheduling performance in a two-cluster system. Simulation Modelling Practice and Theory 17 (2009) 1276–1289
[11] ZHANG Weizhe, FANG Binxing, HU Mingzeng, LIU Xinran, ZHANG Hongli1,GAO Lei1. Multisite co-allocation scheduling algorithms for parallel jobs in computing grid environments. Science in China Series F: Information Sciences 2006 Vol.49 No.6 906—926
[12]王晶,樊晓枉,张盛兵,王海.多核多线程结构线程调度策略研究.计算机科学2007Vol.34No.9
[13]徐斌阳,蒲和平,李少秽.公平调度的分组调度算法设计.电子与信息学报Vbl.30NO.6 Jun.2008
[14]张伟哲,田志宏,张宏莉,何慧,刘文懋.虚拟计算环境中的多机群协同调度算法. Journal of Software, Vol.18, No.8, August 2007, pp.2027?2037
[15] LU Sanglu, ZHOU Xiaobo and XIE Li. A Model for Dynamic Adaptive Coscheduling.计算机科学技术学报1999 Vol.14 No.3
[16] Patrick G.Sobalvarro, Scott Pakin, William E.Weihl and Andrew A.Chien. Dynamic Coscheduling on Workstation Clusters. Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing, p.231-256, March 30, 1998
[17] Patrick Sobalvarro, William E.Weihl. Demand-Based Coscheduling of Parallel Jobs on Multiprogrammed Multiprocessors. Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing, p.106-126, April 25, 1995
[18] Dror G.Feitelson, Larry Rudolph, Uwe Schwiegelshohn, Kenneth C.Sevcik, Parkson Wong. Theory and Practice in Parallel Job Scheduling. April 1997 IPPS'97: Proceedings of the Job Scheduling Strategies for Parallel Processing Publisher: Springer-Verlag
[19]李文隽,万麟瑞.协同任务调度构架模型及关键算法研究.V01.29 No.1 6 Computer Engineering and Design Aug.2008
[20]曹鸿强,卢锡城.基于处理能力预约的工作站集群协同调度.计算机工程与科学Vol.23, No.2, 2001
[21]张小松,刘丹,李毅超.基于服务的两级协同分布式调度策略.Joumal of UEST of China V01.34 No.2 Apr 2005
[22]胡嘉玺.企业级虚拟机应用实战.电子工业出片社. 2009.06
[23] Kenneth J. Duda and David R. Cheriton. Borrowed-Virtual-Time (BVT) scheduling:supporting latency-sensitive threads in a general-purpose scheduler. 1999 ACM 1-58113-140-2/99/0012
[24] Andrea C. Arpaci-Dusseau, David E. Culler, Alan M. Mainwaring. Scheduling with Implicit Information in Distributed Systems. SIGMETRICS’98/PERFORMANCE’98 Joint Conference on the Measurement and Modeling of Computer Systems, June 1998, Madison, Wisconsin.
[25] Deborah A. Wallach, Wilson C. Hsieh, Kirk L. Johnson, M. Frans Kaashoek and William E. Weihl. Optimistic Active Messages: A Mechanism for Scheduling Communication with Computation. PPOPP’95 Santa Clara, CA USA@ 1995 ACM 0-89791-701 -6/95/0007
[26] Andrea Carol Arpaci-Dusseau. Implicit Coscheduling:Coordinated Scheduling with Implicit Information in Distributed Systems. ACM Transactions onComputer Systems, Vol. 9, No.3, August 2001, Pages 283–331.
[27] Gyu Sang Choi, JinHa Kim, Deniz Ersoz, Andy B. Yoo, Chita R.Das. Coscheduling in Clusters: Is It a Viable Alternative?. 0-7695-2153-3/04 (c)2004 IEEE
[28] Shailabh Nagar, Ajit Banerjee, Anand Sivasubramaniam and Chita R. Das. Alternatives to Coscheduling a Network of Workstations1. Journal of Parallel and Distributed Computing 59, 302_327 (1999)
[29] Thorsten von Eicken, David E.Culler, Seth Copen Goldstein, Klaus Erik Schauser. Active Messages: a Mechanism for Integrated Communication and Computation. 1992 ACM 0-89791 -509-7/92/0005/0256
[30] S. Nagar, A. Banerjee, A. Sivasubramaniam, and C. R. Das. A Closer Look at Co-scheduling Approaches for a Network of Workstations. In SPAA’99:Proceedings of the eleventh annual ACM symposium on Parallel algorithms and architectures, pages 96–105, 1999.
[31] J. Nieh and M. Lam. A SMART Scheduler for Multimedia Applications. ACM Transactions on Computer Systems, 21(2):117–163, 2003.
[32] B. Urgaonkar, G. Pacifici, P. Shenoy, M. Spreitzer, and A. Tantawi. An Analytical Model for Multi-tier Internet Services and its Applications. In Proceedings of the ACM International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS 2005), Banff, Canada, June 2005.
[33] Yiyu Chen, Amitayu Das, Anand Sivasubramaniam, Qian Wang, R. Harper, M. Bland. Consolidating Clients on Back-end Servers with Co-location and Frequency Control. Symmetries/Performance’06, June 26–30, 2006, Saint Malo, France. ACM 1-59593-320-4/06/0006.
[34] A. Arpaci-Dusseau and D. E. Culler. Extending Proportional-Share Scheduling to a Network of Workstations. In Proceedings of Parallel and Distributed Processing Techniques and Applications (PDPTA’97), Las Vegas, NV, June 1997.
[35] Kai Shen, Hong Tang, Tao Yang, Lingkun Chu. Integrated Resource Management for Cluster-based Internet Services. In Proceedings of the Fifth USENIX Symposium on Operating Systems Design and Implementation (OSDI), Boston,MA, December 2002.
[36] C. Clark, K. Fraser, Steven Hand, J. Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfield. Live Migration of Virtual Machines. In Proceedings of the Second Symposium on Networked Systems Design and Implementation (NSDI’05),May 2005.
[37] Justin Moore, David Irwin, Laura Grit, Sara Sprenkle and Jeff Chase. Managing Mixed-Use Clusters with Cluster-on-Demand. Technical report, Department of Computer Science, Duke University, November 2002.
[38] Yiyu Chen, Amitayu Das, Wubi Qin, Anand Sivasubramaniam, Qian Wang, Natarajan Gautam. Managing Server Energy and Operational Costs in Hosting Centers. SIGMETRICS’05, June 6–10, 2005, Banff, Alberta, Canada.
[39] Mark S. Squillante, Yanyong Zhang, Anand Sivasubramaniam, Natarajan Gautam, Hubertus Franke, Jose Moreira. Modeling and Analysis of Dynamic Coscheduling in Parallel and Distributed Environments. In SIGMETRICS’02: Proceedings of the 2002 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, pages 43–54, 2002.
[40] Micah Adler, Ying Gong, Arnold L. Rosenberg Optimal. Sharing of Bags of Tasks in Heterogeneous Clusters. SPAA’03, June 7–9, 2003, San Diego, California, USA.
[41] Sameer Shende, Alien D. Malony, Janice Cuny, Peter Beckman, Steve Karmesin, Kathleen Lindlan. Portable Profiling and Tracing for Parallel, Scientific Applications using C++. In Proceedings of ACM SIGMETRICS Symposium on Parallel and Distributed Tools (SPDT), pages 134–145, August 1998
[42] Gaurav Banga, Peter Druschel, Jeffrey C. Mogul. Resource containers: A new facility for resource management in server systems. In Proceedings of the Third Symposium on Operating System Design and Implementation (OSDI’99), New Orleans, pages 45–58, February 1999.
[43] Scott Hazelhurst. Scientific computing using virtual high-performance computing: a case study using the Amazon Elastic Computing Cloud. South African Institute of Computer Scientists and Information Technologists (SAICSIT) Conference, 978-1-60558-286-3, (2008)
[44] Abhishek Chandra, Micah Adler, Pawan Goyaly and Prashant Shenoy. Surplus Fair Scheduling: A Proportional-Share CPU Scheduling Algorithm for Symmetric Multiprocessors. In Proceedings of the Fourth Symposium on Operating System Design and Implementation (OSDI 2000), San Diego, CA, October 2000.
[45] Scott T. Leutenegger ,Mary K. Vernon. The Performance of Multiprogrammed Multiprocessor Scheduling Policies. In SIGMETRICS’90: Proceedings of the 1990 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, pages 226–236, 1990.
[46] Paul Barham, Boris Dragovic, Keir Fraser, Steven Hand, Tim Harris, Alex Ho, Rolf Neugebauer, Ian Pratt, Andrew Warfield. Xen and the Art of Virtualization. SOSP’03, October 19–22, 2003, Bolton Landing, New York, USA
[47] Diwaker Gupta, Rob Gardner, Ludmila Cherkasova. XenMon: QoS Monitoring and Performance Profiling Tool. Technical Report HPL-2005-187, HP Labs, 2005.
[48] James E. Smith, Ravi Nair. Virtual Machines:Versatile Platforms for Systems andProcesses. China Machine Press 2009.03
[49] Intel Corporation 2008,英特尔开源软件技术中心,复旦大学并行处理研究所.系统虚拟化-原理与实现.清华大学出版社2009.03
[50] Ludmila Cherkasova, Diwaker Gupta and Amin Vahdat. Comparison of the Three CPU Schedulers in Xen. ACM SIGMETIUCS Performance Evaluation Review. 2007, 35(2):42-51P
[2] Gine. F, Solsona. F, Hanzich. M, Hernandez. P and Luque. E. Cooperating CoScheduling: A coscheduling proposal aimed at Non-Dedicated Heterogeneous NOWs. JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY 22(5):695～710 Sept.2007
[3] Sriram Govindan, Arjun R. Nath, Amitayu Das, Bhuvan Urgaonkar, Anand Sivasubramaniam. Xen and Co.: Communication-aware CPU Scheduling for Consolidated Xen-based Hosting Platforms. VEE’07, June 13–15, 2007, San Diego, California, USA.
[4] Dror G. Feitelson, Larry Rudolph. Coscheduling Based on Runtime Identification of Activity Working Sets. International Journal of Parallel Programming, Vol. 23, No. 2, 1995
[5] Alexandru Romosan, Doron Rotem, Arie Shoshani and Derek Wright. Co-Scheduling of Computation and Data on Computer Clusters. Proceedings of the 17th international conference on Scientific and statistical database management table of contents.Santa Barbara, CA 2005 ISBN:1-88888-111-X
[6] Dror G. Feitelson, Larry Rudolph. Gang Scheduling Performance Benefits for Fine-Grain Synchronization. Journal of parallel and distributed computing ISSN 0743-7315 1992
[7] Yair Wiseman and Dror G.Feitelson. Paired Gang Scheduling. IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, VOL. 14, NO. 6, JUNE 2003
[8] Dror G. Feitelson, Larry Rudolph. Parallel Job Scheduling—A Status Report. 10th Workshop on Job Scheduling Strategies for Parallel processing Held in conjunction with SIGMETRICS 2004, New York, NY, June 13, 2004
[9] Diego Ongaro, Alan L. Cox, Scott Rixner. Scheduling I/O in Virtual Machine Monitors. VEE’08, March 5–7, 2008, Seattle, Washington, USA
[10] Zafeirios C. Papazachos, Helen D. Karatza. The impact of task service time variability on gang scheduling performance in a two-cluster system. Simulation Modelling Practice and Theory 17 (2009) 1276–1289
[11] ZHANG Weizhe, FANG Binxing, HU Mingzeng, LIU Xinran, ZHANG Hongli1,GAO Lei1. Multisite co-allocation scheduling algorithms for parallel jobs in computing grid environments. Science in China Series F: Information Sciences 2006 Vol.49 No.6 906—926
[12]王晶,樊晓枉,张盛兵,王海.多核多线程结构线程调度策略研究.计算机科学2007Vol.34No.9
[13]徐斌阳,蒲和平,李少秽.公平调度的分组调度算法设计.电子与信息学报Vbl.30NO.6 Jun.2008
[14]张伟哲,田志宏,张宏莉,何慧,刘文懋.虚拟计算环境中的多机群协同调度算法. Journal of Software, Vol.18, No.8, August 2007, pp.2027?2037
[15] LU Sanglu, ZHOU Xiaobo and XIE Li. A Model for Dynamic Adaptive Coscheduling.计算机科学技术学报1999 Vol.14 No.3
[16] Patrick G.Sobalvarro, Scott Pakin, William E.Weihl and Andrew A.Chien. Dynamic Coscheduling on Workstation Clusters. Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing, p.231-256, March 30, 1998
[17] Patrick Sobalvarro, William E.Weihl. Demand-Based Coscheduling of Parallel Jobs on Multiprogrammed Multiprocessors. Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing, p.106-126, April 25, 1995
[18] Dror G.Feitelson, Larry Rudolph, Uwe Schwiegelshohn, Kenneth C.Sevcik, Parkson Wong. Theory and Practice in Parallel Job Scheduling. April 1997 IPPS'97: Proceedings of the Job Scheduling Strategies for Parallel Processing Publisher: Springer-Verlag
[19]李文隽,万麟瑞.协同任务调度构架模型及关键算法研究.V01.29 No.1 6 Computer Engineering and Design Aug.2008
[20]曹鸿强,卢锡城.基于处理能力预约的工作站集群协同调度.计算机工程与科学Vol.23, No.2, 2001
[21]张小松,刘丹,李毅超.基于服务的两级协同分布式调度策略.Joumal of UEST of China V01.34 No.2 Apr 2005
[22]胡嘉玺.企业级虚拟机应用实战.电子工业出片社. 2009.06
[23] Kenneth J. Duda and David R. Cheriton. Borrowed-Virtual-Time (BVT) scheduling:supporting latency-sensitive threads in a general-purpose scheduler. 1999 ACM 1-58113-140-2/99/0012
[24] Andrea C. Arpaci-Dusseau, David E. Culler, Alan M. Mainwaring. Scheduling with Implicit Information in Distributed Systems. SIGMETRICS’98/PERFORMANCE’98 Joint Conference on the Measurement and Modeling of Computer Systems, June 1998, Madison, Wisconsin.
[25] Deborah A. Wallach, Wilson C. Hsieh, Kirk L. Johnson, M. Frans Kaashoek and William E. Weihl. Optimistic Active Messages: A Mechanism for Scheduling Communication with Computation. PPOPP’95 Santa Clara, CA USA@ 1995 ACM 0-89791-701 -6/95/0007
[26] Andrea Carol Arpaci-Dusseau. Implicit Coscheduling:Coordinated Scheduling with Implicit Information in Distributed Systems. ACM Transactions onComputer Systems, Vol. 9, No.3, August 2001, Pages 283–331.
[27] Gyu Sang Choi, JinHa Kim, Deniz Ersoz, Andy B. Yoo, Chita R.Das. Coscheduling in Clusters: Is It a Viable Alternative?. 0-7695-2153-3/04 (c)2004 IEEE
[28] Shailabh Nagar, Ajit Banerjee, Anand Sivasubramaniam and Chita R. Das. Alternatives to Coscheduling a Network of Workstations1. Journal of Parallel and Distributed Computing 59, 302_327 (1999)
[29] Thorsten von Eicken, David E.Culler, Seth Copen Goldstein, Klaus Erik Schauser. Active Messages: a Mechanism for Integrated Communication and Computation. 1992 ACM 0-89791 -509-7/92/0005/0256
[30] S. Nagar, A. Banerjee, A. Sivasubramaniam, and C. R. Das. A Closer Look at Co-scheduling Approaches for a Network of Workstations. In SPAA’99:Proceedings of the eleventh annual ACM symposium on Parallel algorithms and architectures, pages 96–105, 1999.
[31] J. Nieh and M. Lam. A SMART Scheduler for Multimedia Applications. ACM Transactions on Computer Systems, 21(2):117–163, 2003.
[32] B. Urgaonkar, G. Pacifici, P. Shenoy, M. Spreitzer, and A. Tantawi. An Analytical Model for Multi-tier Internet Services and its Applications. In Proceedings of the ACM International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS 2005), Banff, Canada, June 2005.
[33] Yiyu Chen, Amitayu Das, Anand Sivasubramaniam, Qian Wang, R. Harper, M. Bland. Consolidating Clients on Back-end Servers with Co-location and Frequency Control. Symmetries/Performance’06, June 26–30, 2006, Saint Malo, France. ACM 1-59593-320-4/06/0006.
[34] A. Arpaci-Dusseau and D. E. Culler. Extending Proportional-Share Scheduling to a Network of Workstations. In Proceedings of Parallel and Distributed Processing Techniques and Applications (PDPTA’97), Las Vegas, NV, June 1997.
[35] Kai Shen, Hong Tang, Tao Yang, Lingkun Chu. Integrated Resource Management for Cluster-based Internet Services. In Proceedings of the Fifth USENIX Symposium on Operating Systems Design and Implementation (OSDI), Boston,MA, December 2002.
[36] C. Clark, K. Fraser, Steven Hand, J. Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfield. Live Migration of Virtual Machines. In Proceedings of the Second Symposium on Networked Systems Design and Implementation (NSDI’05),May 2005.
[37] Justin Moore, David Irwin, Laura Grit, Sara Sprenkle and Jeff Chase. Managing Mixed-Use Clusters with Cluster-on-Demand. Technical report, Department of Computer Science, Duke University, November 2002.
[38] Yiyu Chen, Amitayu Das, Wubi Qin, Anand Sivasubramaniam, Qian Wang, Natarajan Gautam. Managing Server Energy and Operational Costs in Hosting Centers. SIGMETRICS’05, June 6–10, 2005, Banff, Alberta, Canada.
[39] Mark S. Squillante, Yanyong Zhang, Anand Sivasubramaniam, Natarajan Gautam, Hubertus Franke, Jose Moreira. Modeling and Analysis of Dynamic Coscheduling in Parallel and Distributed Environments. In SIGMETRICS’02: Proceedings of the 2002 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, pages 43–54, 2002.
[40] Micah Adler, Ying Gong, Arnold L. Rosenberg Optimal. Sharing of Bags of Tasks in Heterogeneous Clusters. SPAA’03, June 7–9, 2003, San Diego, California, USA.
[41] Sameer Shende, Alien D. Malony, Janice Cuny, Peter Beckman, Steve Karmesin, Kathleen Lindlan. Portable Profiling and Tracing for Parallel, Scientific Applications using C++. In Proceedings of ACM SIGMETRICS Symposium on Parallel and Distributed Tools (SPDT), pages 134–145, August 1998
[42] Gaurav Banga, Peter Druschel, Jeffrey C. Mogul. Resource containers: A new facility for resource management in server systems. In Proceedings of the Third Symposium on Operating System Design and Implementation (OSDI’99), New Orleans, pages 45–58, February 1999.
[43] Scott Hazelhurst. Scientific computing using virtual high-performance computing: a case study using the Amazon Elastic Computing Cloud. South African Institute of Computer Scientists and Information Technologists (SAICSIT) Conference, 978-1-60558-286-3, (2008)
[44] Abhishek Chandra, Micah Adler, Pawan Goyaly and Prashant Shenoy. Surplus Fair Scheduling: A Proportional-Share CPU Scheduling Algorithm for Symmetric Multiprocessors. In Proceedings of the Fourth Symposium on Operating System Design and Implementation (OSDI 2000), San Diego, CA, October 2000.
[45] Scott T. Leutenegger ,Mary K. Vernon. The Performance of Multiprogrammed Multiprocessor Scheduling Policies. In SIGMETRICS’90: Proceedings of the 1990 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, pages 226–236, 1990.
[46] Paul Barham, Boris Dragovic, Keir Fraser, Steven Hand, Tim Harris, Alex Ho, Rolf Neugebauer, Ian Pratt, Andrew Warfield. Xen and the Art of Virtualization. SOSP’03, October 19–22, 2003, Bolton Landing, New York, USA
[47] Diwaker Gupta, Rob Gardner, Ludmila Cherkasova. XenMon: QoS Monitoring and Performance Profiling Tool. Technical Report HPL-2005-187, HP Labs, 2005.
[48] James E. Smith, Ravi Nair. Virtual Machines:Versatile Platforms for Systems andProcesses. China Machine Press 2009.03
[49] Intel Corporation 2008,英特尔开源软件技术中心,复旦大学并行处理研究所.系统虚拟化-原理与实现.清华大学出版社2009.03
[50] Ludmila Cherkasova, Diwaker Gupta and Amin Vahdat. Comparison of the Three CPU Schedulers in Xen. ACM SIGMETIUCS Performance Evaluation Review. 2007, 35(2):42-51P