SI2-SSE: Enhancement and Support of DMTCP for Adaptive, Extensible Checkpoint-Restart

社会日益复杂的网络基础设施引起了对软件健壮性和可靠性的关注。 然而，同样复杂的基础设施正在威胁容错的继续使用。考虑单个应用程序或硬件设备崩溃的情况。 今天，为了从崩溃的地方恢复应用程序，还必须考虑它所属的复杂子系统。虽然在过去，许多开发人员会编写特定于应用程序的代码来支持单个应用程序的容错，但当重新启动复杂子系统的许多互连应用程序时，这种策略不再可行。 该项目将支持透明检查点重启的插件架构。 透明性意味着软件开发人员不需要编写任何特定于应用程序的代码。插件架构意味着每个软件开发人员只编写一次必要的插件。 每个插件负责恢复一个特定组件的任何中断会话。在更高的层次上，检查点重启系统采用了一系列自主插件，这些插件运行在复杂子系统的所有应用程序上，而不需要任何特定于应用程序的代码。插件架构是一种更通用的方法的一部分，称为流程虚拟化，其中流程外部的所有子系统都被虚拟化。 它将建立在DMTCP检查点重启系统之上。 流程虚拟化的一个简单示例是id的虚拟化。 一个插件维护一个虚拟化表，并安排进程的应用程序代码只看到虚拟id，而外部世界看到的是真实的id。任何使用这个真实的id的系统调用和库调用都被扩展为在真实的和虚拟id之间转换。在重新启动时，真实的id被更新为最新值，进程内存保持不变，因为它只包含虚拟id。采用进程虚拟化的其他技术包括影子设备驱动程序、记录-重放日志和协议虚拟化。 该研究的一些目标包括对InfiniBand网络、可编程GPU（包括着色器）、虚拟机网络、Hadoop等大数据系统以及Android等移动的计算平台的透明检查点重启支持。

Society's increasingly complex cyberinfrastructure creates a concern for software robustness and reliability. Yet, this same complex infrastructure is threatening the continued use of fault tolerance. Consider when a single application or hardware device crashes. Today, in order to resume that application from the point where it crashed, one must also consider the complex subsystem to which it belongs. While in the past, many developers would write application-specific code to support fault tolerance for a single application, this strategy is no longer feasible when restarting the many inter-connected applications of a complex subsystem. This project will support a plugin architecture for transparent checkpoint-restart. Transparency implies that the software developer does not need to write any application-specific code. The plugin architecture implies that each software developer writes the necessary plugins only once. Each plugin takes responsibility for resuming any interrupted sessions for just one particular component. At a higher level, the checkpoint-restart system employs an ensemble of autonomous plugins operating on all of the applications of a complex subsystem, without any need for application-specific code.The plugin architecture is part of a more general approach called process virtualization, in which all subsystems external to a process are virtualized. It will be built on top of the DMTCP checkpoint-restart system. One simple example of process virtualization is virtualization of ids. A plugin maintains a virtualization table and arranges for the application code of the process to see only virtual ids, while the outside world sees the real id. Any system calls and library calls using this real id are extended to translate between real and virtual id. On restart, the real ids are updated with the latest value, and the process memory remains unmodified, since it contains only virtual ids. Other techniques employing process virtualization include shadow device drivers, record-replay logs, and protocol virtualization. Some targets of the research include transparent checkpoint-restart support for the InfiniBand network, for programmable GPUs (including shaders), for networks of virtual machines, for big data systems such as Hadoop, and for mobile computing platforms such as Android.