Comprehensive Measurement and Analysis of the User-Perceived I/O Performance in a Production Leadership-Class Storage System
Comprehensive Measurement and Analysis of the User-Perceived I/O Performance in a Production Leadership-Class Storage System
复制标题
生产领先级存储系统中用户感知 I/O 性能的综合测量和分析
DOI：
--
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
S. Klasky
中科院分区：
文献类型：
--
作者：
Lipeng Wan;M. Wolf;Feiyi Wang;J. Choi;G. Ostrouchov;S. Klasky
随着运行在高性能计算设备上的科学应用所产生的并行I/O工作负载的规模和强度的增加，理解I/O动态，特别是HPC环境中I/O性能变化和退化的根本原因，对HPC社区来说变得非常重要。在本文中，我们在生产领先级存储系统上进行了广泛的I/O测量测试，以捕获大规模并行I/O的性能变化。分析这些结果及其统计相关性，揭示了一些有价值的见解的存储系统的特点和I/O性能变化的根本原因。此外，我们利用这些发现，并提出了一个I/O中间件设计重构，可以提高并行I/O的性能，通过优化数据条带化和放置。我们的初步评估结果表明，所提出的方法可以减少至少80%的平均每进程写延迟和最大每进程写延迟至少20%。
With the increase of the scale and intensity of the parallel I/O workloads generated by those scientific applications running on high performance computing facilities, understanding the I/O dynamics, especially the root cause of the I/O performance variability and degradation in HPC environment, have become extremely critical to the HPC community. In this paper, we run extensive I/O measuring tests on a production leadership-class storage system to capture the performance variabilities of large-scale parallel I/O. Analyzing these results and its statistic correlation revealed some valuable insights into the characteristics of the storage system and the root cause of I/O performance variability. Further, we leverage these findings and propose an I/O middleware design refactoring which can improve the performance of the parallel I/O by optimizing the data striping and placement. Our preliminary evaluation results demonstrate the proposed approach can reduce the average per-process write latency by at least 80% and the maximum per-process write latency by at least 20%.