Enabling portable I/O analysis of commercially sensitive HPC applications through workload replication
Enabling portable I/O analysis of commercially sensitive HPC applications through workload replication
复制标题
通过工作负载复制实现商业敏感 HPC 应用程序的便携式 I/O 分析
DOI：
--
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
S. Jarvis
中科院分区：
文献类型：
--
作者：
J. Dickson;Steven A. Wright;S. Maheswaran;J. Herdman;D. Harris;Mark C. Miller;S. Jarvis
有必要对高性能计算 (HPC) 平台的 I/O 性能进行基准测试和分析，以识别性能瓶颈并指导有效使用新的和现有的存储系统。对大型生产应用程序执行此操作通常具有商业敏感性且缺乏可移植性，这并不是一项简单的任务，并且 I/O 工作负载的代表性代理的可用性可以帮助提供解决方案。我们使用 Darshan I/O 表征和 MACSio 代理应用程序来复制五个生产工作负载，展示了在从小型本地集群到领先规模机器的 HPC 系统之间迁移时如何有效地使用这些工作负载来研究 I/O 性能。初步结果表明，可以生成与目标应用程序匹配的高精度数据集。这使得能够在五个不同的系统上对代表性工作负载进行预测性能分析研究。此分析的结果用于确定工作负载如何在文件系统上表现出不同的 I/O 占用空间以及文件系统配置对性能的影响。
Benchmarking and analyzing I/O performance across high performance computing (HPC) platforms is necessary to identify performance bottlenecks and guide effective use of new and existing storage systems. Doing this with large production applications, which can often be commercially sensitive and lack portability, is not a straightforward task and the availability of a representative proxy for I/O workloads can help to provide a solution. We use Darshan I/O characterization and the MACSio proxy application to replicate five production workloads, showing how these can be used effectively to investigate I/O performance when migrating between HPC systems ranging from small local clusters to leadership scale machines. Preliminary results indicate that it is possible to generate datasets that match the target application with a good degree of accuracy. This enables a predictive performance analysis study of a representative workload to be conducted on five different systems. The results of this analysis are used to identify how workloads exhibit different I/O footprints on a file system and what effect file system configuration can have on performance.