Fine-Grained Policy-Driven I/O Sharing for Burst Buffers
Fine-Grained Policy-Driven I/O Sharing for Burst Buffers
复制标题
DOI：
10.1145/3581784.3607041
复制
发表时间：
2023-06
期刊：
影响因子：
--
通讯作者：
E. Karrels;Lei Huang;Yuhong Kan;Ishank Arora;Yinzhi Wang;D. Katz;W. Gropp;Zhao Zhang
中科院分区：
文献类型：
--
作者：
E. Karrels;Lei Huang;Yuhong Kan;Ishank Arora;Yinzhi Wang;D. Katz;W. Gropp;Zhao Zhang
突发缓冲区是一种常见方法，可以弥合现代超级计算应用程序的I/O需求与大规模超级计算机上共享文件系统的性能之间的性能差距。但是，现有的I/O共享方法需要资源隔离，离线分析或重复执行，从而大大限制了这些系统的利用率和适用性。在这里，我们提出了Themisio，这是一个由政策驱动的I/O共享框架，用于远程共享的爆发缓冲区：专用组的I/O节点组，每个组都带有本地存储设备。 Themisio通过实施机会公平来保留高利用率，以便它可以将未使用的I/O资源重新分配给其他应用程序。 themisio准确有效地在应用程序之间分配I/O周期，纯粹基于实时I/O行为，而无需使用用户提供的信息或脱机的应用程序特征。 Themisio支持各种公平的共享策略，例如用户对，尺寸 - 费用以及综合策略，例如群组 - 用户 - 用户。所有这些功能都通过其统计令牌设计启用。 themisio可以根据分配的令牌更改输入I/O请求的执行顺序，以通过时间切片在应用程序之间精确平衡I/O周期，从而强制执行处理隔离。使用I/O基准测试的实验表明，与现有算法相比，Themisio的I/O吞吐量高13.5-13.7％，其性能变化低19.5-40.4％。对于实际应用，Themisio将速度大幅减少了59.1-99.8％，这是由I/O干扰引起的。
A burst buffer is a common method to bridge the performance gap between the I/O needs of modern supercomputing applications and the performance of the shared file system on large-scale supercomputers. However, existing I/O sharing methods require resource isolation, offline profiling, or repeated execution that significantly limit the utilization and applicability of these systems. Here we present ThemisIO, a policy-driven I/O sharing framework for a remote-shared burst buffer: a dedicated group of I/O nodes, each with a local storage device. ThemisIO preserves high utilization by implementing opportunity fairness so that it can reallocate unused I/O resources to other applications. ThemisIO accurately and efficiently allocates I/O cycles among applications, purely based on real-time I/O behavior without requiring user-supplied information or offline-profiled application characteristics. ThemisIO supports a variety of fair sharing policies, such as user-fair, size-fair, as well as composite policies, e.g., group-then-user-fair. All these features are enabled by its statistical token design. ThemisIO can alter the execution order of incoming I/O requests based on assigned tokens to precisely balance I/O cycles between applications via time slicing, thereby enforcing processing isolation. Experiments using I/O benchmarks show that ThemisIO sustains 13.5-13.7% higher I/O throughput and 19.5-40.4% lower performance variation than existing algorithms. For real applications, ThemisIO significantly reduces the slowdown by 59.1-99.8% caused by I/O interference.