Streamlining High-End Computing with Software Persistent Memory

当前的高端计算（HEC）应用程序显式地管理持久数据，包括应用程序状态和应用程序输出。这种做法不仅增加了开发时间和成本，而且还要求应用程序开发人员密切了解与平台相关的底层存储机制，以实现良好的应用程序I/O性能。这种垂直开发也使得应用软件的可移植性降低。软件持久内存（SoftPM）项目为下一代HEC应用程序简化数据管理构建了一个轻量级基础设施。SoftPM允许应用程序以与易失性内存相同的方式分配持久内存，从而消除了HEC应用程序中数据管理的二元性，并且可以轻松地恢复、浏览和与过去版本的持久内存状态进行交互。这简化了HEC应用程序所需的三种广泛功能的实现-可恢复性（例如，检查点重新启动），记录重播（例如，数据可视化）和执行分支（例如，模拟模型-空间探索）。SoftPM项目被组织成三个模块。第一个模块构建了一个可演进的SoftPM API，并解决了内存管理问题。第二个模块处理本地存储和并行文件系统的高性能I/O和持久点的原子性。最后一个模块构建了几个HEC应用案例研究，以说明SoftPM在HEC环境中支持的不同功能。

Current high-end computing (HEC) applications explicitly manage persistent data, including both application state and application output. This practice not only increases development time and cost, but also requires an application developer to be intimately aware of the underlying platform-dependent storage mechanisms to achieve good application I/O performance. Such vertical development also makes the application software less portable.The Software Persistent Memory (SoftPM) project builds a lightweight infrastructure for streamlining data management in next generation HEC applications. SoftPM eliminates the duality of data management in HEC applications by allowing applications to allocate persistent memory in much the same way volatile memory is allocated and easily restore, browse, and interact with past versions of persistent memory state. This simplifies the implementation of three broad capabilities required in HEC applications -- recoverability (e.g., checkpoint-restart), record-replay (e.g., data-visualization), and execution branching (e.g., simulation model-space exploration).The SoftPM project is organized in three modules. The first module builds an evolvable SoftPM API and addresses memory management issues. The second module addresses high-performance I/O and the atomicity of persistence points for local storage and parallel file systems. The final module builds several HEC application case-studies to illustrate the different capabilities supported by SoftPM in HEC environments.