NGS: A System for Semi-Automatic Application-Level Checkpointing of Parallel Programs

摘要：CNS-0406345执行摘要：该项目将设计和实现一个预编译器和运行时系统，该系统将使现有的并行C或Fortran程序（使用MPI或OpenMp库调用实现并行性）能够检查点并恢复其计算状态。 与Condor等系统采用的检查点不同，这些系统产生的检查点是可移植的，因为它们可以用于在另一台具有不同处理器类型和配置的机器上重新启动应用程序。 该项目的目标是在对应用程序的性能和可伸缩性影响最小的情况下实现这一点。 该项目将开发，而大型机器上同一应用程序的完整系统级检查点将需要将TB级数据保存到磁盘。 在网格环境中，系统级检查点所采用的检查点是不可移植的，因此计算不能迁移以利用不断变化的资源可用性。目前，希望使用应用程序级检查点的程序员必须手动分析和检测他们的代码。 我们的提案系统将自动化这一点，只需要程序员的最小输入。 实现这一点需要新的检查点协议来协调进程，其中一些我们已经开发和实现了。 它还需要过程间程序分析技术和一个复杂的运行时系统，我们将在这个项目中实现。 我们采用的方法需要高性能计算、程序分析运行时系统设计和分布式计算方面的专业知识。 我们相信我们的团队能够迎接挑战。

Abstract: CNS-0406345Executive Summary:The project will design and implement a pre-compiler and runtime system that will enable an existing parallel C or Fortran program, which uses MPI or OpenMp library calls for parallelism, to checkpoint and restore its computational state. Unlike the checkpoints taken by a system like Condor, the checkpoints produced by these system will be portable in the sense that they can be used to restart the application on another machine of a direrent processor type and conguration. The projects goal is to accomplish this with minimal impact on the performance and scalability of the application. The project will develop, whereas a complete system-level checkpoint for the same application on a large machine will need to save terabytes of data to disk. In a grid environment, the checkpoints taken by system-level checkpointing are not portable, so computations cannot migrate to take advantage of changing resource availability.At present, programmers who wish to use application-level checkpointing must analyze and instrument their code manually. Our proposal system will automate this, requiring only minimal input from the programmer. Accomplishing this requires new checkpoint protocols for process coordination, some of which we have already developed and implemented. It also requires inter-procedural program analysis techniques and a sophisticated runtime system, which we will implement in this project. The approach we are taking requires expertise in high-performance computing, program analysis runtime system design, and distributed computing. We believe our project team can rise to the challenge.