Hierarchical redesign of classic MPI reduction algorithms

详细信息    查看全文

• 作者：Khalid Hasanov ; Alexey Lastovetsky
• 关键词：MPI collectives ; Reduction ; Hierarchical MPI
• 刊名：The Journal of Supercomputing
• 出版年：2017
• 出版时间：February 2017
• 年：2017
• 卷：73
• 期：2
• 页码：713-725
• 全文大小：
• 刊物类别：Computer Science
• 刊物主题：Programming Languages, Compilers, Interpreters; Processor Architectures; Computer Science, general;
• 出版者：Springer US
• ISSN：1573-0484
• 卷排序：73

文摘

Optimization of MPI collective communication operations has been an active research topic since the advent of MPI in 1990s. Many general and architecture-specific collective algorithms have been proposed and implemented in the state-of-the-art MPI implementations. Hierarchical topology-oblivious transformation of existing communication algorithms has been recently proposed as a new promising approach to optimization of MPI collective communication algorithms and MPI-based applications. This approach has been successfully applied to the most popular parallel matrix multiplication algorithm, SUMMA, and the state-of-the-art MPI broadcast algorithms, demonstrating significant multifold performance gains, especially for large-scale HPC systems. In this paper, we apply this approach to optimization of the MPI Reduce and Allreduce operations. Theoretical analysis and experimental results on a cluster of Grid’5000 platform are presented.