CNS Core: Small: REYAZ: Reliability-Aware Job Scheduling for HPC Systems

高性能计算（HPC）使不同科学领域的突破能够改善国民经济，健康，福利和国防。不幸的是，高性能计算系统提供生产性科学的能力现在开始受到硬件相关错误和故障的严重阻碍。因此，具有国家重要性的计算应用将需要在弹性机制中花费大量执行时间，以便在存在故障的情况下取得进展。尽管如此，由于应用程序执行过程中的故障中断频率很高，未来的HPC系统将浪费大量的资源和时间。为了应对这些挑战，这个名为REYAZ的项目探索了高性能计算作业调度的新领域：通过联合利用系统组件的动态可靠性状态和应用程序的弹性特征，最大限度地提高可靠性受限的高性能计算系统上完成的有用工作量。REYAZ将实现两种新功能：(1)可靠性感知作业调度方法，在保证单个应用程序“公平”性能的同时，在不可靠的大规模计算系统上优化单位时间内完成的有用工作。(2)一系列减少输入/输出（I/O）开销的技术，这是广泛使用的弹性机制（如检查点重新启动）的副作用，同时保留通过可靠性感知调度获得的性能改进。在未来受可靠性限制的高性能计算系统中，单位时间内的有效工作最大化将直接转化为更高效的科学——导致不同科学领域和社会影响的更快发展。该项目开发的能力还将有助于减少大型系统上的能源浪费，从而为社会带来经济效益。该项目将把研究任务和成果整合到教育活动中，以培养下一代工程师，他们将面对操作不可靠的大型系统的挑战。来自代表性不足群体的本科生将参与并接受大规模容错并行计算领域的培训。项目网站（https://github.com/GoodwillComputingLab/REYAZ）将提供作为项目一部分开发的所有研究成果和软件工件的文档，包括系统软件、运行时系统、分析工具、建模方法、实验数据和跟踪。项目网站将在项目结束后至少五年积极维护。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

High-performance computing (HPC) enables breakthroughs in different science domains that lead to an improvement in national economics, health, welfare, and defense. Unfortunately, the ability of HPC systems to deliver productive science is now beginning to be significantly hampered by hardware-related errors and failures. Consequently, computational applications of national importance will need to spend a large fraction of execution time in resilience mechanisms to make forward progress in the presence of failures. Despite that, a huge amount of resources and time will be wasted on future HPC systems due to the high frequency of failure interruptions during application execution. To address these challenges, this project, called REYAZ, explores new territory in HPC job scheduling: maximizing the amount of useful work done on reliability-constrained HPC systems by jointly exploiting dynamic reliability state of the system components and resilience characteristics of applications. REYAZ will enable two novel capabilities: (1) a reliability-aware job scheduling approach that optimizes useful work done per unit time on unreliable large-scale computing systems while individual applications are guaranteed "fair" performance. (2) a family of techniques to reduce the input/output (I/O) overhead - a side-effect of widely used resilience mechanisms such as checkpoint-restart - while retaining the performance improvements obtained via reliability-aware scheduling. Maximizing the useful work per unit time on future reliability-constrained HPC systems will directly translate into more productive science - leading to faster advancements of different science fields and societal impact. Capabilities developed in this project will also help reduce the wastage of energy on large-scale systems resulting in economic benefits for the society. This project will integrate the research tasks and outcomes into educational activities to train the next generation of engineers who will face the challenges of operating unreliable large-scale systems. Undergraduate students from underrepresented groups will be engaged and trained in the field of large-scale fault-tolerant parallel computing.The project website (https://github.com/GoodwillComputingLab/REYAZ) will host all the documentation of research findings and software artifacts developed as a part of the project, including system software, runtime systems, analytical tools, modeling methodologies, experimental data, and traces. The project website will be maintained actively for at least five years beyond the project end date.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.