HECURA: Collaborative Research: QoS-driven Storage Management for High-end Computing Systems

在当今的高端计算（HEC）系统中，并行文件系统（PFS）是存储基础架构的核心。PFS部署由许多用户和应用程序共享，但目前没有区分服务的规定-数据访问以尽力而为的方式提供。随着系统的扩展，这种限制可能会阻止应用程序在实现其期望性能的同时有效地利用HEC资源，并且它会阻碍并发地支持大量数据密集型应用程序。该项目旨在解决服务质量驱动的HEC存储管理中的挑战，旨在通过解决以下四个方面的研究来支持PFS中的I/O带宽保证：1。基于PFS虚拟化的每个应用程序I/O带宽分配，其中每个应用程序通过其动态创建的虚拟PFS获得其特定的I/O带宽份额。2. PFS管理服务，控制每个应用程序虚拟PFS的生命周期和配置，并支持应用程序I/O监控和存储资源预留。3.通过跨应用程序的自主细粒度资源调度实现高效的I/O带宽分配，这些应用程序包含基于分析和预测的协调调度和优化。4.可扩展的应用程序检查点设置，基于针对检查点设置I/O定制的虚拟PFS上的性能隔离和优化。

In today's high-end computing (HEC) systems, the parallel file system (PFS) is at the core of the storage infrastructure. PFS deployments are shared by many users and applications, but currently there are no provisions for differentiation of service - data access is provided in a best-effort manner. As systems scale, this limitation can prevent applications from efficiently utilizing the HEC resources while achieving their desired performance and it presents a hurdle to support a large number of data-intensive applications concurrently. This NSF HECURA project tackles the challenges in quality of service (QoS) driven HEC storage management, aiming to support I/O bandwidth guarantees in PFSs by addressing the following four research aspects: 1. Per-application I/O bandwidth allocation based on PFS virtualization, where each application gets its specific I/O bandwidth share through its dynamically created virtual PFS. 2. PFS management services that control the lifecycle and configuration of per-application virtual PFSs as well as support application I/O monitoring and storage resource reservation. 3. Efficient I/O bandwidth allocation through autonomic, fine-grained resource scheduling across applications that incorporate coordinated scheduling and optimizations based on profiling and prediction. 4. Scalable application checkpointing based on performance isolation and optimization on virtual PFSs customized for checkpointing I/Os.