HECURA: Collaborative Research: QoS-driven Storage Management for High-end Computing Systems

在当今的高端计算（HEC）系统中，并行文件系统（PFS）是存储基础设施的核心。PFS部署由许多用户和应用程序共享，但目前还没有区分服务的规定——数据访问是以尽力而为的方式提供的。随着系统的扩展，这一限制可能会阻碍应用程序在实现预期性能的同时有效地利用HEC资源，并为并发支持大量数据密集型应用程序带来障碍。NSF HECURA项目旨在解决服务质量（QoS）驱动的HEC存储管理方面的挑战，旨在通过以下四个研究方面支持pfs中的I/O带宽保证：基于PFS虚拟化的每个应用程序I/O带宽分配，其中每个应用程序通过其动态创建的虚拟PFS获得其特定的I/O带宽共享。2. PFS管理服务，控制每个应用程序虚拟PFS的生命周期和配置，并支持应用程序I/O监控和存储资源预留。3. 通过跨应用程序的自主的、细粒度的资源调度来实现高效的I/O带宽分配，这些应用程序结合了基于分析和预测的协调调度和优化。4. 基于为检查点I/ o定制的虚拟pfs的性能隔离和优化的可扩展应用程序检查点。

In today's high-end computing (HEC) systems, the parallel file system (PFS) is at the core of the storage infrastructure. PFS deployments are shared by many users and applications, but currently there are no provisions for differentiation of service - data access is provided in a best-effort manner. As systems scale, this limitation can prevent applications from efficiently utilizing the HEC resources while achieving their desired performance and it presents a hurdle to support a large number of data-intensive applications concurrently. This NSF HECURA project tackles the challenges in quality of service (QoS) driven HEC storage management, aiming to support I/O bandwidth guarantees in PFSs by addressing the following four research aspects: 1. Per-application I/O bandwidth allocation based on PFS virtualization, where each application gets its specific I/O bandwidth share through its dynamically created virtual PFS. 2. PFS management services that control the lifecycle and configuration of per-application virtual PFSs as well as support application I/O monitoring and storage resource reservation. 3. Efficient I/O bandwidth allocation through autonomic, fine-grained resource scheduling across applications that incorporate coordinated scheduling and optimizations based on profiling and prediction. 4. Scalable application checkpointing based on performance isolation and optimization on virtual PFSs customized for checkpointing I/Os.