SI2-SSI: Collaborative Research: A Software Infrastructure for MPI Performance Engineering: Integrating MVAPICH and TAU via the MPI Tools Interface

消息传递接口(MPI)继续在超级计算领域占据主导地位，成为首选的主要并行编程模型。当今使用的大量科学应用都基于MPI。在当前和下一代高端计算(HEC)系统上，必须了解时间关键型应用程序和底层MPI实施之间的交互，以便更好地优化它们的可扩展性和性能。HEC系统的当前用户使用高性能MPI实现开发他们的应用程序，但使用独立的性能工具分析和微调行为。从本质上讲，每个软件组件都将对方视为一个黑匣子，很少共享信息或访问可能对优化策略有用的功能。缺乏允许分析工具和MPI库之间交互的标准化接口一直是一个很大的障碍。MPI-3标准中新引入的MPI_T接口提供了一种简单的机制，允许MPI实现者从实现中公开表示配置参数或性能测量的变量，以利于工具、调优框架和其他支持库。然而，很少有性能分析和调优工具利用MPI_T接口，也没有一个这样做来在执行时动态优化。这项研究和开发工作旨在利用新的MPI_T接口来构建MPI性能工程的软件基础设施。随着MPI标准中MPI_T的采用，现在有可能采取积极的步骤来实现MPI库和性能工具之间的紧密交互和集成。这项研究由来自OSU和UO的计算机科学家团队负责，他们代表开源的MVAPICH和TAU项目，旨在创建一个基于MPI_T接口的开源集成软件基础设施，该接口定义了用于交互和信息交换的API，以实现高性能计算应用程序的细粒度性能优化。该项目解决的挑战包括：1)增强MVAPICH中对MPI_T的现有支持，以公开更丰富的性能和控制变量集；2)重新设计TAU，以利用MVAPICH所公开的新的MPI_T变量；3)扩展和增强TAU和MVAPICH，使其能够生成建议和性能工程报告；4)提出基本的设计更改，以使MVAPICH等MPI库在运行时“可重新配置”；以及5)为交互式性能工程会话添加对MVAPICH和TAU的支持。该框架将在各种高性能计算基准和应用程序上进行验证。集成的中间件和工具将向社区公开提供。这项研究将对实现以前难以提供的高性能计算应用程序的优化产生重大影响。因此，它将有助于得出在下一代多千万亿次浮点和亿级系统上运行的“最佳实践”指南。研究方向及其解决方案将用于PIS的课程，以培养本科生和研究生。

Message-Passing Interface (MPI) continues to dominate the supercomputing landscape, being the primary parallel programming model of choice. A large variety of scientific applications in use today are based on MPI. On the current and next-generation High-End Computing (HEC) systems, it is essential to understand the interaction between time-critical applications and the underlying MPI implementations in order to better optimize them for both scalability and performance. Current users of HEC systems develop their applications with high-performance MPI implementations, but analyze and fine tune the behavior using standalone performance tools. Essentially, each software component views the other as a blackbox, with little sharing of information or access to capabilities that might be useful in optimization strategies. Lack of a standardized interface that allows interaction between the profiling tool and the MPI library has been a big impediment. The newly introduced MPI_T interface in the MPI-3 standard provides a simple mechanism that allows MPI implementers to expose variables representing configuration parameters or performance measurements from within the implementation for the benefit of tools, tuning frameworks, and other support libraries. However, few performance analysis and tuning tools take advantage of the MPI_T interface and none do so to dynamically optimize at execution time. This research and development effort aims to build a software infrastructure for MPI performance engineering using the new MPI_T interface.With the adoption of MPI_T in the MPI standard, it is now possible to take positive steps to realize close interaction between and integration of MPI libraries and performance tools. This research, undertaken by a team of computer scientists from OSU and UO representing the open source MVAPICH and TAU projects, aims to create an open source integrated software infrastructure built on the MPI_T interface which defines the API for interaction and information interchange to enable fine grained performance optimizations for HPC applications. The challenges addressed by the project include: 1) enhancing existing support for MPI_T in MVAPICH to expose a richer set of performance and control variables; 2) redesigning TAU to take advantage of the new MPI_T variables exposed by MVAPICH; 3) extending and enhancing TAU and MVAPICH with the ability to generate recommendations and performance engineering reports; 4) proposing fundamental design changes to make MPI libraries like MVAPICH ``reconfigurable'' at runtime; and 5) adding support to MVAPICH and TAU for interactive performance engineering sessions. The framework will be validated on a variety of HPC benchmarks and applications. The integrated middleware and tools will be made publicly available to the community. The research will have a significant impact on enabling optimizations of HPC applications that have previously been difficult to provide. As a result, it will contribute to deriving "best practice" guidelines for running on next-generation Multi-Petaflop and Exascale systems. The research directions and their solutions will be used in the curriculum of the PIs to train undergraduate and graduate students.