高性能并行计算系统中低功耗资源管理的设计与研究
|
摘要
当前，高性能计算被广泛应用于高科技研究和诸多工业领域，以高性能计算为基础的计算科学得到了显著的发展。同时，需求的不断增加和对性能的苛刻要求给新一代的高性能计算系统的研究与设计带来了严峻的挑战。特别是随着处理器性能和系统规模的迅速提升，急剧增长的功耗严重限制了高性能计算系统的设计和使用。为有效管理系统功耗，提高系统的可靠性和可用性，最终降低高性能计算系统的拥有代价，低功耗技术已成为高性能计算领域的关键技术。
上世纪90年代以来，大规模并行处理系统（massively parallel processing，MPP）和集群（Cluster）系统已成为高性能计算机主要的体系结构。在并行计算系统中，并行资源管理软件根据调度算法从作业队列中选择合适的作业，并为其分配和释放计算结点。传统的作业调度和资源管理主要关注两点：降低作业的平均等待时间和提高整个系统的利用率。然而，由于没有足够的作业负载，以及作业调度和资源分配策略无法充分利用系统资源，常常会有空闲资源浪费系统能耗的情况。
针对以上问题，本课题分析了高性能并行计算系统负载与能耗的特点，并基于并行资源管理技术，设计了两类针对高性能并行计算系统的自适应功耗管理算法。两类功耗管理算法分别采用了限制资源分配和关闭空闲结点的方法。其中基于限制资源分配的算法根据系统利用率或作业平均slowdown的变化，自适应调整可用结点数目，并关闭非可用结点以节省系统能耗；关闭空闲结点的算法则根据结点的关闭间隔时间或请求和服务速率自适应调整结点的空闲时间阀值，并使空闲时间超出阀值的结点进入“睡眠”状态以节省系统能耗。实验使用来源于ParallelWorkloads Archive的负载对两类算法进行了测试，结果表明，两类算法均能在不违反性能限制的前提下，有效降低系统能耗。
Nowadays high-performance computing has been widely used in high-tech research and numerous industrial fields. The field of High-performance computing has experienced extensive changes and remarkable development. At the same time, the sharp increase in power consumption has posed a serious challenge to the reliability, availability, and usability of high-performance computing systems.
Since the 90s of last century, massively parallel processing (MPP) and clusters have become the two major architectures in high-performance computing systems. In parallel computers, resource management software is responsible for job scheduling and resource allocation. Scheduling policies in parallel systems normally focuses on how to reduce the average job waiting time and improve overall system utilization. However, because there is not enough load, as well as job scheduling policies can not make full use of system’resources, there are significant time periods during which a great many computing nodes in system cannot be utilized.
To solve the above problems, this paper analyzes the characteristics of workload and power usage in high- performance parallel computing system. Based on this analysis, we propose two types of self-adaptive power management algorithms to reduce the energy consumption. The first algorithm adjusts the number of available nodes according to the system utilization or job average slowdown while power off unavailable nodes. The second algorithm optimizes the energy efficiency of system by control the node’s threshold of idle time period. A node will be switched into“sleep”mode if it’s idle time period longer than threshold. Detailed experimentation using traces from the Parallel Workloads Archive indicates that there algorithms can achieve considerable overall system energy savings without violating the performance constraints.
上世纪90年代以来，大规模并行处理系统（massively parallel processing，MPP）和集群（Cluster）系统已成为高性能计算机主要的体系结构。在并行计算系统中，并行资源管理软件根据调度算法从作业队列中选择合适的作业，并为其分配和释放计算结点。传统的作业调度和资源管理主要关注两点：降低作业的平均等待时间和提高整个系统的利用率。然而，由于没有足够的作业负载，以及作业调度和资源分配策略无法充分利用系统资源，常常会有空闲资源浪费系统能耗的情况。
针对以上问题，本课题分析了高性能并行计算系统负载与能耗的特点，并基于并行资源管理技术，设计了两类针对高性能并行计算系统的自适应功耗管理算法。两类功耗管理算法分别采用了限制资源分配和关闭空闲结点的方法。其中基于限制资源分配的算法根据系统利用率或作业平均slowdown的变化，自适应调整可用结点数目，并关闭非可用结点以节省系统能耗；关闭空闲结点的算法则根据结点的关闭间隔时间或请求和服务速率自适应调整结点的空闲时间阀值，并使空闲时间超出阀值的结点进入“睡眠”状态以节省系统能耗。实验使用来源于ParallelWorkloads Archive的负载对两类算法进行了测试，结果表明，两类算法均能在不违反性能限制的前提下，有效降低系统能耗。
Nowadays high-performance computing has been widely used in high-tech research and numerous industrial fields. The field of High-performance computing has experienced extensive changes and remarkable development. At the same time, the sharp increase in power consumption has posed a serious challenge to the reliability, availability, and usability of high-performance computing systems.
Since the 90s of last century, massively parallel processing (MPP) and clusters have become the two major architectures in high-performance computing systems. In parallel computers, resource management software is responsible for job scheduling and resource allocation. Scheduling policies in parallel systems normally focuses on how to reduce the average job waiting time and improve overall system utilization. However, because there is not enough load, as well as job scheduling policies can not make full use of system’resources, there are significant time periods during which a great many computing nodes in system cannot be utilized.
To solve the above problems, this paper analyzes the characteristics of workload and power usage in high- performance parallel computing system. Based on this analysis, we propose two types of self-adaptive power management algorithms to reduce the energy consumption. The first algorithm adjusts the number of available nodes according to the system utilization or job average slowdown while power off unavailable nodes. The second algorithm optimizes the energy efficiency of system by control the node’s threshold of idle time period. A node will be switched into“sleep”mode if it’s idle time period longer than threshold. Detailed experimentation using traces from the Parallel Workloads Archive indicates that there algorithms can achieve considerable overall system energy savings without violating the performance constraints.
引文
[1] High Productivity Computer Systems. http://www.highproductivity.org/, 2009.
[2] ReliaSoft research team. Arrhenius Relationship Introduction. http://www.weibull.com/AccelTestWeb/arrhenius_relationship_chap_.htm, 1997.
[3] Sushant Sharma, Chung-Hsing Hsu, Wu-chun Feng. Making a Case for a Green500 List. http://www.green500.org/resources.php, 2006.
[4] Moore. Cramming more components onto integrated circuits. Electronics, 1965, 38.
[5] W. Feng. Making a case for efficient supercomputing. ACM Queue, 2003, 1(7):54~64.
[6] Hans Werner Meuer. The TOP500 Project: Looking Back over 15 Years of Supercomputing. Informatik-Spektrum, 2008, 31(3): 203~222.
[7] TOP 500 supercomputer sites. http://www.top500.org/overtime, 2009.
[8] Wu-chun Feng, Kirk W. Cameron. The Green500 List: Encouraging Sustainable Supercomputing. IEEE Computer, 2007, 40:50~55.
[9] Jed Scaramella. Worldwide Server Power and Cooling Expense 2006-2010 Forecast. IBM Whitepapers, 2007.
[10] John Humphreys, Jed Scaramella. The Impact of Power and Cooling on Data Center Infrastructure. IDC market research report, 2006.
[11] Rong Ge, Xizhou Feng, Kirk W. Cameron. Improvement of Power-Performance Efficiency for High-End Computing. Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium, 2005.
[12] W. Feng. The Importance of Being Low Power in High-Performance Computing. Cyberinfrastructure Technology Watch, 2005, 1:12~21.
[13] Wikipedia. Green Computing. http://en.wikipedia.org/wiki/Green_computing, 2009.
[14] Energy Star. http://www.energystar.gov/, 2009.
[15] Mike Ebbers, Alvin Galea, Michael Schaefer, Marc Tu Duy Khiem. The Green Data Center: Steps for the Journey. IBM Redpaper, 2008.
[16] The Green Grid. http://www.thegreengrid.org/, 2009.
[17] The Green500 list. http://www.green500.org/, 2009.
[18] Wu-chun Feng and Chung-hsing Hsu. Green Destiny and its Evolving Parts. Innovative Supercomputer Architecture Award, International Supercomputer Conference , 2004.
[19] W. Feng and C. Hsu. The Origin and Evolution of Green Destiny. InProc. of IEEE Cool Chips VII: An International Symposium on LowPower and High Speed Chips, 2004.
[20] SLURM: A Highly Scalable Resource Manager. https://computing.llnl.gov/linux/slurm/slurm.html, 2009.
[21] Parallel Workloads Archive. http://www.cs.huji.ac.il/labs/parallel/workload/, 2009.
[22] TOP 500 supercomputer sites. http://www.top500.org/lists/2009/06/performance _development, 2009.
[23] Lei Qiao. Current and Future HPC Architectures. http://www.ece.rochester.edu /~qiao/docs/urfile/supercomputer_qiao.pdf, 2007.
[24] Eerth Simulator. http://www.jamstec.go.jp/es/en/es1/system/index.html, 2009.
[25] Eerth Simulator System Overview. http://www.jamstec.go.jp/es/en/es1/system/system.html, 2009.
[26] The BlueGene/L Team. An Overview of the BlueGene/L Supercomputer. Proceedings of the 2002 ACM/IEEE conference on Supercomputing, 2002:1~22.
[27] Wikipedia BlueGene/L. http://en.wikipedia.org/wiki/File:BlueGeneL_schema.png, 2009.
[28]顾跃,魏玉琪. IBM Blue Gene/P计算机结构分析.高性能计算发展与应用. 2008, (24):10~13.
[29] Beowulf HOWTO. http://www.ibiblio.org/pub/linux/docs/HOWTO/archive/Beowulf-HOWTO.html#ss2.2, 2009.
[30] Wikipedia Beowulf. http://en.wikipedia.org/wiki/File:Beowulf.png, 2009.
[31] Vasanth Venkatachalam, Michael Franz. Power Reduction Techniques For Microprocessor Systems, ACM Computing Surveys, 2005, 37(3):95~237.
[32] Fen Xie, Margaret Martonosi, Sharad Malik. Compile Time Dynamic Voltage Scaling Settings: Opportunities and Limits. Proceedings of the ACM SIGPLAN 2003 conference on Programming language design and implementation, 2003:49~62.
[33] ACPI Working Group. ACPI specification Version 3.0b. http://www.acpi.info/spec.htm, 2006.
[34] AMD Corporation. Mobile AMD Duron Processor Model 7 Data Sheet. AMD, 2001.
[35] Intel Corporation. Dual-Core Intel Xeon Processor LV and ULV Data sheet. Intel, 2006.
[36] K.-Y. Mun, D.-W. Kim, D.-H. Kim, C.-I. Park. DVS: An efficient dynamic voltage scaling algorithm based on the differential of CPU utilization. In The 9th Asia-Pacific Computer Systems Architecture Conference (ACSAC), 2004.
[37] Rong Ge, Kirk W. Cameron. Power-Aware Speedup. IEEE International Parallel and Distributed Processing Symposium, 2007.
[38] Bruce R. Childers, Hongliang Tang, Rami Melhem. Adapting Processor Supply Voltage to Instruction-Level Parallelism. Microprocessors White Papers, 2001.
[39] P. P′enzes, A. Martin. Energy-delay efficiency of VLSI computations. In The ACM Great Lakes Symposium on VLSI (GLSVLSI), 2002.
[40] The Green Grid. The Green Grid Data Center Power Efficiency Metrics: PUE and DCiE. Technical Committee White Paper, http://www.thegreengrid.org/gg_content/TGG_Data_Center_Power_Efficiency_Metrics_PUE_and_DCiE.pdf, 2007.
[41] Hongbo Yang. Power-aware Compilation Techniques for High Performance Processors. Doctor dissertation, University of Delaware, 2004.
[42] ChungHsing Hsu, Ulrich Kremer. The Design, Implementation, and Evaluation of a Compiler Algorithm for CPU Energy Reduction. Proceedings of the ACM SIGPLAN 2003 conference on Programming language design and implementation, 2003:38~48.
[43] ChungHsing Hsu. Compiler-directed Dynamic Voltage And Frequency Scaling for CPU Power and Energy Reduction. University of New Brunswick, 2003.
[44] K.-Y. Mun, D.-W. Kim, D.-H. Kim, C.-I. Park. DVS: An efficient dynamic voltage scaling algorithm based on the differential of CPU utilization. In The 9th Asia-Pacific Computer Systems Architecture Conference (ACSAC), 2004.
[45] Andreas Weissel, Frank Bellosa. Process cruise control: event-driven clock scaling for dynamic power management. In Proceedings of the International Conference on Compilers, Architecture, and Synthesis for Embedded Systems CASES’02, 2002.
[46] F. Bellosa, A. Weissel, M. Waitz, S. Kellner. Event–driven energy accounting for dynamic thermal management. In Proceedings of the Workshop on Compilers and Operating Systems for Low Power (COLP’03), 2003.
[47] W. L. Bircher, J. Law, M. Valluri, L. K. John. Effective Use of Performance Monitoring Counters for Run-Time Prediction of Power, University of Texas at Austin, 2004.
[48] Rong Ge, Xizhou Feng, Wu-chun Feng, Kirk W. Cameron, CPU MISER: A Performance-Directed, Run-Time System for Power-Aware Clusters. International Conference on Parallel Processing (ICPP 07), 2007.
[49] Jian Li. Power-Performance Considerations of Parallel Computing on Chip Multiprocessors. ACM Transactions on Architecture and Code Optimization (TACO), 2005, 2(4):397~422.
[50] Murali Annavaram, Ed Grochowski, John Shen. Mitigating Amdahl’s Law Through EPI Throttling. ACM SIGARCH Computer Architecture News, 2005, 33(2):298~209.
[51] Ryan E. Grant. Ahmad Afsahi, Power-Performance Efficiency of Asymmetric Multiprocessors for Multi-threaded Scientific Applications. In the 20th International Parallel and Distributed Processing Symposium, 2006.
[52] Barry Lawson, Evgenia Smirni, Power-aware Resource Allocation in High-end Systems via Online Simulation. Proceedings of the 19th annual international conference on Supercomputing, 2005, (6):229~238.
[53] Andre Kerstens. Steven A. DuChene, Applying Green Computing to clusters and the data center. Proceedings of the Linux Symposium, http://ols.fedoraproject.org/OLS/Reprints-2008/duchene-reprint.pdf, 2008.
[2] ReliaSoft research team. Arrhenius Relationship Introduction. http://www.weibull.com/AccelTestWeb/arrhenius_relationship_chap_.htm, 1997.
[3] Sushant Sharma, Chung-Hsing Hsu, Wu-chun Feng. Making a Case for a Green500 List. http://www.green500.org/resources.php, 2006.
[4] Moore. Cramming more components onto integrated circuits. Electronics, 1965, 38.
[5] W. Feng. Making a case for efficient supercomputing. ACM Queue, 2003, 1(7):54~64.
[6] Hans Werner Meuer. The TOP500 Project: Looking Back over 15 Years of Supercomputing. Informatik-Spektrum, 2008, 31(3): 203~222.
[7] TOP 500 supercomputer sites. http://www.top500.org/overtime, 2009.
[8] Wu-chun Feng, Kirk W. Cameron. The Green500 List: Encouraging Sustainable Supercomputing. IEEE Computer, 2007, 40:50~55.
[9] Jed Scaramella. Worldwide Server Power and Cooling Expense 2006-2010 Forecast. IBM Whitepapers, 2007.
[10] John Humphreys, Jed Scaramella. The Impact of Power and Cooling on Data Center Infrastructure. IDC market research report, 2006.
[11] Rong Ge, Xizhou Feng, Kirk W. Cameron. Improvement of Power-Performance Efficiency for High-End Computing. Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium, 2005.
[12] W. Feng. The Importance of Being Low Power in High-Performance Computing. Cyberinfrastructure Technology Watch, 2005, 1:12~21.
[13] Wikipedia. Green Computing. http://en.wikipedia.org/wiki/Green_computing, 2009.
[14] Energy Star. http://www.energystar.gov/, 2009.
[15] Mike Ebbers, Alvin Galea, Michael Schaefer, Marc Tu Duy Khiem. The Green Data Center: Steps for the Journey. IBM Redpaper, 2008.
[16] The Green Grid. http://www.thegreengrid.org/, 2009.
[17] The Green500 list. http://www.green500.org/, 2009.
[18] Wu-chun Feng and Chung-hsing Hsu. Green Destiny and its Evolving Parts. Innovative Supercomputer Architecture Award, International Supercomputer Conference , 2004.
[19] W. Feng and C. Hsu. The Origin and Evolution of Green Destiny. InProc. of IEEE Cool Chips VII: An International Symposium on LowPower and High Speed Chips, 2004.
[20] SLURM: A Highly Scalable Resource Manager. https://computing.llnl.gov/linux/slurm/slurm.html, 2009.
[21] Parallel Workloads Archive. http://www.cs.huji.ac.il/labs/parallel/workload/, 2009.
[22] TOP 500 supercomputer sites. http://www.top500.org/lists/2009/06/performance _development, 2009.
[23] Lei Qiao. Current and Future HPC Architectures. http://www.ece.rochester.edu /~qiao/docs/urfile/supercomputer_qiao.pdf, 2007.
[24] Eerth Simulator. http://www.jamstec.go.jp/es/en/es1/system/index.html, 2009.
[25] Eerth Simulator System Overview. http://www.jamstec.go.jp/es/en/es1/system/system.html, 2009.
[26] The BlueGene/L Team. An Overview of the BlueGene/L Supercomputer. Proceedings of the 2002 ACM/IEEE conference on Supercomputing, 2002:1~22.
[27] Wikipedia BlueGene/L. http://en.wikipedia.org/wiki/File:BlueGeneL_schema.png, 2009.
[28]顾跃,魏玉琪. IBM Blue Gene/P计算机结构分析.高性能计算发展与应用. 2008, (24):10~13.
[29] Beowulf HOWTO. http://www.ibiblio.org/pub/linux/docs/HOWTO/archive/Beowulf-HOWTO.html#ss2.2, 2009.
[30] Wikipedia Beowulf. http://en.wikipedia.org/wiki/File:Beowulf.png, 2009.
[31] Vasanth Venkatachalam, Michael Franz. Power Reduction Techniques For Microprocessor Systems, ACM Computing Surveys, 2005, 37(3):95~237.
[32] Fen Xie, Margaret Martonosi, Sharad Malik. Compile Time Dynamic Voltage Scaling Settings: Opportunities and Limits. Proceedings of the ACM SIGPLAN 2003 conference on Programming language design and implementation, 2003:49~62.
[33] ACPI Working Group. ACPI specification Version 3.0b. http://www.acpi.info/spec.htm, 2006.
[34] AMD Corporation. Mobile AMD Duron Processor Model 7 Data Sheet. AMD, 2001.
[35] Intel Corporation. Dual-Core Intel Xeon Processor LV and ULV Data sheet. Intel, 2006.
[36] K.-Y. Mun, D.-W. Kim, D.-H. Kim, C.-I. Park. DVS: An efficient dynamic voltage scaling algorithm based on the differential of CPU utilization. In The 9th Asia-Pacific Computer Systems Architecture Conference (ACSAC), 2004.
[37] Rong Ge, Kirk W. Cameron. Power-Aware Speedup. IEEE International Parallel and Distributed Processing Symposium, 2007.
[38] Bruce R. Childers, Hongliang Tang, Rami Melhem. Adapting Processor Supply Voltage to Instruction-Level Parallelism. Microprocessors White Papers, 2001.
[39] P. P′enzes, A. Martin. Energy-delay efficiency of VLSI computations. In The ACM Great Lakes Symposium on VLSI (GLSVLSI), 2002.
[40] The Green Grid. The Green Grid Data Center Power Efficiency Metrics: PUE and DCiE. Technical Committee White Paper, http://www.thegreengrid.org/gg_content/TGG_Data_Center_Power_Efficiency_Metrics_PUE_and_DCiE.pdf, 2007.
[41] Hongbo Yang. Power-aware Compilation Techniques for High Performance Processors. Doctor dissertation, University of Delaware, 2004.
[42] ChungHsing Hsu, Ulrich Kremer. The Design, Implementation, and Evaluation of a Compiler Algorithm for CPU Energy Reduction. Proceedings of the ACM SIGPLAN 2003 conference on Programming language design and implementation, 2003:38~48.
[43] ChungHsing Hsu. Compiler-directed Dynamic Voltage And Frequency Scaling for CPU Power and Energy Reduction. University of New Brunswick, 2003.
[44] K.-Y. Mun, D.-W. Kim, D.-H. Kim, C.-I. Park. DVS: An efficient dynamic voltage scaling algorithm based on the differential of CPU utilization. In The 9th Asia-Pacific Computer Systems Architecture Conference (ACSAC), 2004.
[45] Andreas Weissel, Frank Bellosa. Process cruise control: event-driven clock scaling for dynamic power management. In Proceedings of the International Conference on Compilers, Architecture, and Synthesis for Embedded Systems CASES’02, 2002.
[46] F. Bellosa, A. Weissel, M. Waitz, S. Kellner. Event–driven energy accounting for dynamic thermal management. In Proceedings of the Workshop on Compilers and Operating Systems for Low Power (COLP’03), 2003.
[47] W. L. Bircher, J. Law, M. Valluri, L. K. John. Effective Use of Performance Monitoring Counters for Run-Time Prediction of Power, University of Texas at Austin, 2004.
[48] Rong Ge, Xizhou Feng, Wu-chun Feng, Kirk W. Cameron, CPU MISER: A Performance-Directed, Run-Time System for Power-Aware Clusters. International Conference on Parallel Processing (ICPP 07), 2007.
[49] Jian Li. Power-Performance Considerations of Parallel Computing on Chip Multiprocessors. ACM Transactions on Architecture and Code Optimization (TACO), 2005, 2(4):397~422.
[50] Murali Annavaram, Ed Grochowski, John Shen. Mitigating Amdahl’s Law Through EPI Throttling. ACM SIGARCH Computer Architecture News, 2005, 33(2):298~209.
[51] Ryan E. Grant. Ahmad Afsahi, Power-Performance Efficiency of Asymmetric Multiprocessors for Multi-threaded Scientific Applications. In the 20th International Parallel and Distributed Processing Symposium, 2006.
[52] Barry Lawson, Evgenia Smirni, Power-aware Resource Allocation in High-end Systems via Online Simulation. Proceedings of the 19th annual international conference on Supercomputing, 2005, (6):229~238.
[53] Andre Kerstens. Steven A. DuChene, Applying Green Computing to clusters and the data center. Proceedings of the Linux Symposium, http://ols.fedoraproject.org/OLS/Reprints-2008/duchene-reprint.pdf, 2008.