SPX: Collaborative Research: Cross-layer Application-Aware Resilience at Extreme Scale (CAARES)

SPX：协作研究：超大规模跨层应用程序感知弹性 (CAARES)

• 批准号: 1725499
• 负责人: Barbara Chapman
• 金额: $ 26.67万
• 依托单位: SUNY at Stony Brook
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-08-15 至 2021-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1725499&HistoricalAwards=false
• 关键词: SPX; Collaborative; Research; Cross; layer

The increasing demands of science and engineering applications push the limits of current large-scale systems, and is expected to achieve exascale (10^18 FLOPS) performance early in the next decade. One of the lesser studied challenge at extreme scales is the reliability of the computing system itself, primarily due to the very large number of cores and components utilized and to the sharp decrease of the Mean Time Between Failures on such systems (in the order of tens of minutes). This project departs from the traditional single component fault management model, and explores how multiple software libraries (and application components) used in the context of a single parallel application can interact to provide the holistic fault management support necessary for parallel applications targeting capability computing. This exploration will not be limited to software developed using a single parallel programming paradigm, but will be extended to encompass the more challenging case where multiple programming paradigms can be combined to achieve a common goal, to simulate a set of large scale scientific applications in use today.  The goal of this project is to depart from the current siloed resilience mechanisms, and propose cross-layer composition solutions that can fundamentally address these resilience challenges at extreme scales. This exploration will not be limited to software developed using a single parallel programming paradigm, but will be extended to encompass the more challenging case where multiple programming paradigms can be combined to achieve a common goal, to simulate a set of large scale scientific applications in use today. More specifically, this proposal will address the following research challenges: (1) development of a theoretical foundation for a deeper understanding of the challenges and opportunities arising from combining different resilience models and methodologies; (2) design of a flexible programming abstraction to allow different resilience models and mechanisms to be combined to cooperate and address resilience in a more holistic manner; and (3) development of basic, programming paradigm independent, constructs necessary to implement cross-layer and domain-specific approaches to support resilience and to understand related performance / quality trade-offs. The proposed approach will be validated by exposing these generic abstractions in two different programming paradigms (MPI and OpenSHMEM), by creating and developing specialized concepts for each of these paradigms. This will enable the assessment of the validity of the concepts and the corresponding overheads imposed by the different software layers, using few software frameworks and applications.

科学和工程应用的日益增长的需求推动了当前大规模系统的极限，预计在下个十年初将达到艾级(10^18 Flop)的性能。在极端规模下较少研究的挑战之一是计算系统本身的可靠性，这主要是因为使用了非常多的核心和部件，以及这种系统的平均无故障时间急剧减少(大约几十分钟)。该项目不同于传统的单组件故障管理模型，探讨了在单个并行应用程序环境中使用的多个软件库(和应用程序组件)如何交互，以提供面向能力计算的并行应用程序所需的整体故障管理支持。这一探索将不限于使用单一并行编程范例开发的软件，而是将扩展到包括更具挑战性的情况，其中多个编程范例可以被组合以实现共同的目标，以模拟当今使用的一组大规模科学应用。本项目的目标是背离当前孤立的弹性机制，并提出能够在极端规模上从根本上解决这些弹性挑战的跨层组合解决方案。这一探索将不限于使用单一并行编程范例开发的软件，而将被扩展以涵盖更具挑战性的情况，其中多个编程范例可以组合以实现共同的目标，来模拟当今正在使用的一组大规模科学应用程序。更具体地说，这项提议将解决以下研究挑战：(1)为更深入地了解结合不同复原力模型和方法所产生的挑战和机遇奠定理论基础；(2)设计灵活的方案编制抽象，以便能够结合不同的复原力模型和机制，以更全面的方式合作和解决复原力问题；(3)开发独立于编程范式的基本结构，以实施支持复原力和了解相关性能/质量权衡的跨层和具体领域的方法。建议的方法将通过在两个不同的编程范例(MPI和OpenSHMEM)中公开这些通用抽象，通过为每个范例创建和开发专门的概念来验证。这将能够使用较少的软件框架和应用程序来评估概念的有效性和不同软件层施加的相应管理费用。

项目成果

OpenSHMEM Checker - A Clang Based Static Checker for OpenSHMEM

• DOI: 10.1109/ispdc52870.2021.9521645
• 发表时间: 2021-07
• 期刊: 2021 20th International Symposium on Parallel and Distributed Computing (ISPDC)
• 作者: Md Abdullah Shahneous Bari;Ujjwal Arora;Varun Hegde;Tony Curtis;B. Chapman

Checkpointing OpenSHMEM Programs Using Compiler Analysis

使用编译器分析对 OpenSHMEM 程序进行检查点

• DOI: 10.1109/ftxs51974.2020.00011
• 发表时间: 2020
• 期刊: 2020 IEEE/ACM 10th Workshop on Fault Tolerance for HPC at eXtreme Scale (FTXS
• 作者: Shahneous Bari, Md Abdullah;Basu, Debasmita;Lu, Wenbin;Curtis, Tony;Chapman, Barbara
• 通讯作者: Chapman, Barbara