SHF: Medium: Collaborative Research: Next-Generation Message Passing for Parallel Programming: Resiliency, Time-to-Solution, Performance-Portability, Scalability, and QoS

基于MPI的并行编程在学术界、政府(国防和非国防用途)以及可扩展机器学习和大数据分析的新兴用途中的使用频率正在增加。新兴的超级计算机系统将有更多的故障，MPI需要能够解决这些故障，以适应这些新出现的情况，而不是导致整个应用程序失败。高性能计算(HPC)的协作性、变革性消息传递研究对新的和即将到来的可伸缩系统中的性能可移植并行编程至关重要(其战略是“先最佳实践，后标准化”)正在被简化为实践。消息传递接口(MPI-3/4)应用程序编程接口的一个重要子集是通过具有在并行任务之间同步的弱集体事务的扩展来实现容错的。这项研究研究了一种新的模型，它定位故障，提供可调的无故障开销，允许多种类型的故障，支持分层恢复，并且与数据并行相关。底层网络的故障建模正在研究中。在这项工作中，应用程序开发人员可以控制粒度和无故障开销。中间件原型的性能和可伸缩性结果主要是通过紧凑的应用程序来演示的，这些应用程序与具有实际和学术意义的实际用例相关。这项工作的影响范围从政府实验室中最大的超级计算机的用户，到拥有长时间运行的时间关键型应用程序的实用集群，以及在比过去几年更频繁发生故障的“恶劣”环境中的空基和其他并行处理。该项目正在生产可用的自由软件，这些软件将在社区中广泛分享，并指导学术界、工业界和政府如何编写更好的并行程序。该项目还提供了如何更新现有或遗留程序以使用正在减少到实践中的新功能的指导方针。

Parallel programming based on MPI is being used with increased frequency in academia, government (defense and non-defense uses), as well as emerging uses in scalable machine learning and big data analytics. Emerging supercomputer systems will have more faults and MPI needs to be able to workaround such faults to be appropriate to these emerging situations, rather than causing an entire application to fail. Collaborative, transformative message passing research for High Performance Computing (HPC) critical to performance-portable parallel programming in new and forthcoming scalable systems (with a strategy of "best practice-first, standardization-later") is being reduced to practice. A substantial subset of the Message Passing Interface (MPI-3/4) application programmer interface is being made fault tolerant through extensions with weak collective transactions that synchronize between parallel tasks. This research studies the novel model that localizes faults, provides tunable fault-free overhead, allows for multiple kinds of faults, enables hierarchical recovery, and is data-parallel relevant. Fault modeling of underlying networks is being studied. Application developers control the granularity and fault-free overhead in this effort. Performance and scalability results of the middleware prototype are being demonstrated principally through compact applications that relate to real use cases of practical and academic interest. The impact of this work ranges from users of the largest supercomputers in government labs to practical clusters that have long-running, time-critical applications, and to space-based and other parallel processing in "hostile" environments where faults occur more frequently than in past years. The project is producing usable free software that will be widely shared in the community as well as guidance on how better parallel programs can be written in academia, industry, and government. The project also provides guidelines for how to update existing or legacy programs to use the new capabilities that are being reduced to practice.