SI2-SSE: Enhancement and Support of DMTCP for Adaptive, Extensible Checkpoint-Restart

SI2-SSE：DMTCP 的增强和支持，以实现自适应、可扩展的检查点重启

• 批准号: 1440788
• 负责人: Gene Cooperman
• 金额: $ 49.92万
• 依托单位: Northeastern University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-09-01 至 2018-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1440788&HistoricalAwards=false
• 关键词: SI2; SSE; Enhancement; Support; DMTCP

Society's increasingly complex cyberinfrastructure creates a concern for software robustness and reliability. Yet, this same complex infrastructure is threatening the continued use of fault tolerance. Consider when a single application or hardware device crashes. Today, in order to resume that application from the point where it crashed, one must also consider the complex subsystem to which it belongs. While in the past, many developers would write application-specific code to support fault tolerance for a single application, this strategy is no longer feasible when restarting the many inter-connected applications of a complex subsystem. This project will support a plugin architecture for transparent checkpoint-restart. Transparency implies that the software developer does not need to write any application-specific code. The plugin architecture implies that each software developer writes the necessary plugins only once. Each plugin takes responsibility for resuming any interrupted sessions for just one particular component. At a higher level, the checkpoint-restart system employs an ensemble of autonomous plugins operating on all of the applications of a complex subsystem, without any need for application-specific code.The plugin architecture is part of a more general approach called process virtualization, in which all subsystems external to a process are virtualized. It will be built on top of the DMTCP checkpoint-restart system. One simple example of process virtualization is virtualization of ids. A plugin maintains a virtualization table and arranges for the application code of the process to see only virtual ids, while the outside world sees the real id. Any system calls and library calls using this real id are extended to translate between real and virtual id. On restart, the real ids are updated with the latest value, and the process memory remains unmodified, since it contains only virtual ids. Other techniques employing process virtualization include shadow device drivers, record-replay logs, and protocol virtualization. Some targets of the research include transparent checkpoint-restart support for the InfiniBand network, for programmable GPUs (including shaders), for networks of virtual machines, for big data systems such as Hadoop, and for mobile computing platforms such as Android.

社会日益复杂的网络基础设施引发了对软件健壮性和可靠性的担忧。然而，同样复杂的基础设施正在威胁容错的持续使用。考虑单个应用程序或硬件设备崩溃的情况。今天，为了从该应用程序崩溃的地方恢复它，还必须考虑它所属的复杂子系统。虽然在过去，许多开发人员会编写特定于应用程序的代码来支持单个应用程序的容错，但在重新启动复杂子系统的许多相互连接的应用程序时，这种策略不再可行。该项目将支持透明检查点重启的插件架构。透明意味着软件开发人员不需要编写任何特定于应用程序的代码。插件架构意味着每个软件开发人员只需编写一次必要的插件。每个插件负责为一个特定组件恢复任何中断的会话。在更高的层次上，检查点重新启动系统采用一组在复杂子系统的所有应用程序上运行的自主插件，而不需要任何特定于应用程序的代码。插件架构是一种更通用的称为流程虚拟化的方法的一部分，在这种方法中，流程外部的所有子系统都被虚拟化。它将建立在DMTCP检查点重新启动系统之上。流程虚拟化的一个简单示例是id的虚拟化。插件维护一个虚拟化表，并安排进程的应用程序代码只看到虚拟id，而外部世界看到真实id。使用这个实id的任何系统调用和库调用都被扩展为在实id和虚id之间进行转换。在重新启动时，实际id被更新为最新的值，进程内存保持不变，因为它只包含虚拟id。其他使用进程虚拟化的技术包括影子设备驱动程序、记录重放日志和协议虚拟化。该研究的一些目标包括对InfiniBand网络、可编程gpu（包括着色器）、虚拟机网络、大数据系统（如Hadoop）和移动计算平台（如Android）的透明检查点重启支持。

项目成果

CRUM: Checkpoint-Restart Support for CUDA's Unified Memory

• DOI: 10.1109/cluster.2018.00047
• 发表时间: 2018-08
• 期刊: 2018 IEEE International Conference on Cluster Computing (CLUSTER)
• 作者: Rohan Garg;Apoorve Mohan;Michael B. Sullivan;G. Cooperman