NGS: A System for Semi-Automatic Application-Level Checkpointing of Parallel Programs

摘要：CNS-0406345执行摘要：该项目将设计和实现一个预编译器和运行时系统，该系统将使现有的使用MPI或OpenMp库调用并行的C或Fortran程序能够检查点并恢复其计算状态。与秃鹰这样的系统获取的检查点不同，这些系统生成的检查点将是可移植的，因为它们可以用于在另一台处理器类型和拥塞程度不同的机器上重新启动应用程序。项目的目标是在对应用程序的性能和可伸缩性影响最小的情况下实现这一点。该项目将进行开发，而大型计算机上同一应用程序的完整系统级检查点将需要将数TB的数据保存到磁盘。在网格环境中，系统级检查点获取的检查点是不可移植的，因此计算无法迁移以利用不断变化的资源可用性。目前，希望使用应用级检查点的程序员必须手动分析和插入代码。我们的提案系统将自动执行此操作，只需要程序员提供最少的输入。要做到这一点，需要为进程协调制定新的检查点协议，我们已经开发和实施了其中一些协议。它还需要过程间程序分析技术和复杂的运行时系统，这是我们将在本项目中实现的。我们采用的方法需要高性能计算、程序分析运行时系统设计和分布式计算方面的专业知识。我们相信我们的项目团队能够迎接挑战。

Abstract: CNS-0406345Executive Summary:The project will design and implement a pre-compiler and runtime system that will enable an existing parallel C or Fortran program, which uses MPI or OpenMp library calls for parallelism, to checkpoint and restore its computational state. Unlike the checkpoints taken by a system like Condor, the checkpoints produced by these system will be portable in the sense that they can be used to restart the application on another machine of a direrent processor type and conguration. The projects goal is to accomplish this with minimal impact on the performance and scalability of the application. The project will develop, whereas a complete system-level checkpoint for the same application on a large machine will need to save terabytes of data to disk. In a grid environment, the checkpoints taken by system-level checkpointing are not portable, so computations cannot migrate to take advantage of changing resource availability.At present, programmers who wish to use application-level checkpointing must analyze and instrument their code manually. Our proposal system will automate this, requiring only minimal input from the programmer. Accomplishing this requires new checkpoint protocols for process coordination, some of which we have already developed and implemented. It also requires inter-procedural program analysis techniques and a sophisticated runtime system, which we will implement in this project. The approach we are taking requires expertise in high-performance computing, program analysis runtime system design, and distributed computing. We believe our project team can rise to the challenge.