Collaborative Research: SI2-SSI: EVOLVE: Enhancing the Open MPI Software for Next Generation Architectures and Applications

合作研究：SI2-SSI：EVOLVE：增强下一代架构和应用的开放式 MPI 软件

• 批准号: 1664142
• 负责人: George Bosilca
• 金额: $ 156.62万
• 依托单位: University of Tennessee Knoxville
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-06-01 至 2022-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1664142&HistoricalAwards=false
• 关键词: Collaborative; Research; SI2; SSI; EVOLVE

For nearly two decades, the Message Passing Interface (MPI) has been an essential part of the High-Performance Computing ecosystem and consequently a key enabler for important scientific breakthroughs. It is a fundamental building block for most large-scale simulations from physics, chemistry, biology, material sciences as engineering. Open MPI is an open source implementation of the MPI specification, widely used and adopted by the research community as well as industry. The Open MPI library is jointly developed and maintained by a consortium of academic institutions, national labs and industrial partners. It is installed on virtually all large-scale computer systems in the US as well as in the rest of the world. The goal of this project is to enhance and modernize the Open MPI library in the context of the ongoing evolution of modern computer systems, and to ensure its future operability on all upcoming architectures. We aim at implementing fundamental software techniques that can be used in many-core systems to execute MPI-based parallel applications more efficiently, and to tolerate process and memory failures at all scales, from current systems, up to the extreme scales expected before the end of the decade.Open MPI is an open source implementation of the Message Passing Interface (MPI) specification. The MPI API is currently being extended to consider the needs of application developers in terms of efficiency, productivity and resilience. The project will also support academic involvement in the design, development and evaluation of the Open MPI software, and ensure academic presence in the MPI Forum. The goal of this proposal is to enhance the Open MPI software library, focusing on two aspects: (1) Extend Open MPI to support new features of the MPI specification. Open MPI will continue to support all new features of current and upcoming MPI specifications. The two most significant areas within the context of this proposal are (a) extensions to better support hybrid programming models and (b) support for fault tolerance in MPI applications. To improve support for hybrid programming models, the MPI Forum is currently considering introducing the notion of MPI Endpoints, which could be used by different threads of an MPI rank to instantiate multiple separate communication contexts. The goal within this project is to develop an implementation of endpoints to support effective hybrid programming model, and to extend the concept to other aspects of parallel applications such as File I/O operations. One of the project partners (UTK) leads the current proposal in the MPI Forum to expose failures and ensure the continuation of the execution of MPI applications. In the context of this SSI proposal, the goal is to harden, improve, and expand the support of the existing ULFM implementation in Open MPI and thus enable end-users to design application-specific resilience approaches for future platforms. (2) Enhance the Open MPI core to support new architectures and improve scalability. While Open MPI has demonstrated very good scalability in the past, there is significant work to be done to ensure similarly good performance on future architectures. Specifically, we propose a groundbreaking rework of the startup environment that will improve process launch scalability, increase support for asynchronous progress of operations, enable support for accelerators, and reduce sensitivity to system noise. The project would also enhance the support for File I/O operations as part of the Open MPI package by expanding our work on highly scalable collective I/O operations through delegation and exploring the utilization of burst buffers as temporary storage.

近二十年来，消息传递接口（MPI）一直是高性能计算生态系统的重要组成部分，因此也是重要科学突破的关键推动因素。它是物理学、化学、生物学、材料科学和工程学等大多数大规模模拟的基本构建块。 Open MPI是MPI规范的开源实现，被研究界和工业界广泛使用和采用。Open MPI库由学术机构、国家实验室和工业合作伙伴联合开发和维护。它几乎安装在美国和世界其他地区的所有大型计算机系统上。该项目的目标是在现代计算机系统不断发展的背景下增强和现代化Open MPI库，并确保其未来在所有即将到来的架构上的可操作性。我们的目标是实现基本的软件技术，可用于众核系统，以执行基于MPI的并行应用程序更有效地，并容忍在所有规模的进程和内存故障，从目前的系统，到极端的规模预计在十年结束之前。开放MPI是消息传递接口（MPI）规范的开源实现。MPI API目前正在扩展，以考虑应用程序开发人员在效率、生产力和弹性方面的需求。该项目还将支持学术界参与开放式MPI软件的设计、开发和评估，并确保学术界参与MPI论坛。本提案的目标是增强Open MPI软件库，主要集中在两个方面：（1）扩展Open MPI以支持MPI规范的新功能。Open MPI将继续支持当前和即将推出的MPI规范的所有新功能。在这个建议的上下文中，两个最重要的领域是（a）扩展以更好地支持混合编程模型和（B）支持MPI应用程序中的容错。为了提高对混合编程模型的支持，MPI论坛目前正在考虑引入MPI端点的概念，MPI端点可以被MPI等级的不同线程用来实例化多个单独的通信上下文。该项目的目标是开发一个端点实现，以支持有效的混合编程模型，并将该概念扩展到并行应用程序的其他方面，如文件I/O操作。项目合作伙伴之一（UTK）在MPI论坛中领导了当前的提案，以暴露故障并确保MPI应用程序的继续执行。在这个SSI提案的背景下，目标是加强、改进和扩展Open MPI中现有ULFM实现的支持，从而使最终用户能够为未来的平台设计特定于应用程序的弹性方法。(2)增强Open MPI核心以支持新架构并提高可扩展性。虽然Open MPI在过去已经展示了非常好的可扩展性，但要确保在未来的架构上具有类似的良好性能，还有大量的工作要做。具体来说，我们提出了一个开创性的启动环境的返工，这将提高流程启动的可扩展性，增加对异步操作进度的支持，支持加速器，并降低对系统噪声的敏感性。该项目还将增强对文件I/O操作的支持，作为Open MPI包的一部分，通过委托扩展我们在高度可扩展的集体I/O操作方面的工作，并探索利用突发缓冲区作为临时存储。

项目成果

A failure detector for HPC platforms

• DOI: 10.1177/1094342017711505
• 发表时间: 2018-01-01
• 期刊: INTERNATIONAL JOURNAL OF HIGH PERFORMANCE COMPUTING APPLICATIONS
• 影响因子: 3.1
• 作者: Bosilca, George;Bouteiller, Aurelien;Dongarra, Jack
• 通讯作者: Dongarra, Jack

Using Advanced Vector Extensions AVX-512 for MPI Reductions

使用高级矢量扩展 AVX-512 减少 MPI

• DOI: 10.1145/3416315.3416316
• 发表时间: 2020
• 期刊: EuroMPI/USA '20: 27th European MPI Users' Group Meeting
• 作者: Zhong, Dong;Cao, Qinglei;Bosilca, George;Dongarra, Jack
• 通讯作者: Dongarra, Jack

Predicting MPI Collective Communication Performance Using Machine Learning

使用机器学习预测 MPI 集体通信性能

• DOI: 10.1109/cluster49012.2020.00036
• 发表时间: 2020
• 期刊: 2020 IEEE International Conference on Cluster Computing (CLUSTER
• 作者: Hunold, Sascha;Bhatele, Abhinav;Bosilca, George;Knees, Peter
• 通讯作者: Knees, Peter

ADAPT: an event-based adaptive collective communication framework

• DOI: 10.1145/3208040.3208054
• 发表时间: 2018-06
• 期刊: Proceedings of the 27th International Symposium on High-Performance Parallel and Distributed Computing
• 作者: Xi Luo;Wei Wu;G. Bosilca;Thananon Patinyasakdikul;Linnan Wang;J. Dongarra

DeepFreeze: Towards Scalable Asynchronous Checkpointing of Deep Learning Models

DeepFreeze：迈向深度学习模型的可扩展异步检查点

• DOI: 10.1109/ccgrid49817.2020.00-76
• 发表时间: 2020
• 期刊: Cloud and Internet Computing (CCGRID
• 作者: Nicolae, B.;Li, J.;Wozniak, J..;Bosilca, G.;Dorier, M.;Cappello, F.
• 通讯作者: Cappello, F.