Modular HPC I/O Characterization with Darshan
Modular HPC I/O Characterization with Darshan
复制标题
使用 Darshan 进行模块化 HPC I/O 表征
DOI：
10.1109/espt.2016.9
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
N. Wright
中科院分区：
文献类型：
--
作者：
Shane Snyder;P. Carns;K. Harms;R. Ross;Glenn K. Lockwood;N. Wright
当代高性能计算（HPC）应用程序包含广泛的不同I/O策略，并且在其生命周期中通常在许多不同的计算平台上执行。这些大规模HPC平台采用越来越复杂的I/O子系统来为应用程序提供合适的I/O性能。为这样的系统调优I/O工作负载是非常重要的，并且结果通常不能移植到其他HPC系统。I/O分析工具可以帮助解决这一挑战，但是大多数现有的工具只能检测I/O子系统中的特定组件，这些组件只能提供有限的I/O性能视图。科学应用和计算平台的日益多样化要求I/O表征具有更大的灵活性和范围。在这项工作中，我们考虑如何改进I/O分析工具Darshan，以便对当前和未来的HPC I/O工作负载进行更灵活、更全面的检测。我们评估了我们设计的性能和可扩展性，以确保它足够轻量级，可以在生产HPC系统上进行全职部署。我们还提供了两个案例研究，说明更全面的应用程序I/O工作负载检测如何能够深入了解以前无法实现的I/O行为。我们的结果表明，Darshan的模块化检测方法可以为用户和系统管理员提供有价值的反馈，同时对用户应用程序的开销可以忽略。
Contemporary high-performance computing (HPC) applications encompass a broad range of distinct I/O strategies and are often executed on a number of different compute platforms in their lifetime. These large-scale HPC platforms employ increasingly complex I/O subsystems to provide a suitable level of I/O performance to applications. Tuning I/O workloads for such a system is nontrivial, and the results generally are not portable to other HPC systems. I/O profiling tools can help to address this challenge, but most existing tools only instrument specific components within the I/O subsystem that provide a limited perspective on I/O performance. The increasing diversity of scientific applications and computing platforms calls for greater flexibility and scope in I/O characterization.In this work, we consider how the I/O profiling tool Darshan can be improved to allow for more flexible, comprehensive instrumentation of current and future HPC I/O workloads. We evaluate the performance and scalability of our design to ensure that it is lightweight enough for full-time deployment on production HPC systems. We also present two case studies illustrating how a more comprehensive instrumentation of application I/O workloads can enable insights into I/O behavior that were not previously possible. Our results indicate that Darshan's modular instrumentation methods can provide valuable feedback to both users and system administrators, while imposing negligible overheads on user applications.