Collaborative Research: Personalized Benchmarks for High Performance Computing Applications

随着高性能计算应用程序针对越来越大的问题，数据输入和输出（I/O）占用越来越多的运行时间。用户、软件开发人员和平台管理员经常发现很难理解应用程序的I/O代码在做什么，为什么它很慢，如何改进，或者它在不同平台上的性能如何。I/O基准测试有助于解决这个问题，但是它们的生产成本很高，因此不适用于大多数应用程序。通过利用现有的轻量级I/O分析器，该项目为所有应用程序提供了用户友好的个性化I/O基准测试，这些I/O分析器已经在高性能计算平台上监视应用程序的行为。由此产生的个性化基准将帮助研究人员、开发人员和购买者评估潜在的新存储系统架构，评估现有或新版本的存储系统和I/O库，规划购买，比较应用程序集群或跨平台工作负载的性能，以及提高并行I/O库和应用程序的性能。分析和基准生成软件以及示例基准将会公开发布。这个项目使用两种方法来构建个性化的I/O基准。首先，该项目通过提供分析和可视化工具，将每次运行期间由轻量级I/O分析器（如Darshan）自动捕获的信息传达给涉众，从而使现有应用程序在所有运行期间进行自我基准测试。其次，该项目正在创建平台定制的基准套件，这些套件表示在给定平台上观察到的应用程序级工作负载的组合。为了实现这一点，该项目根据生产作业的I/O行为对其进行集群化，并使用新的和现有的I/O内核生成技术为每个集群生成一个紧凑的基准测试。由此产生的基准测试套件将作为实际的、特定于平台的生产I/O工作负载的代理，并提供以前无法获得的关于这些工作负载在给定设施中的普遍程度的见解，从而提高技术水平。

As high-performance computing applications target ever-larger problems, data input and output (I/O) takes up more and more run time. Users, software developers, and platform administrators often find it difficult to understand what an application's I/O code is doing, why it is slow, how it might be improved, or how well it would perform on a different platform. I/O benchmarks help address this problem, but they are expensive to produce and thus are not available for most applications. This project is providing user-friendly personalized I/O benchmarks for all applications, by leveraging existing lightweight I/O profilers that already monitor the behavior of applications on high-performance computing platforms. The resulting personalized benchmarks will help researchers, developers, and purchasers in evaluating potential new storage system architectures, evaluating existing or new versions of storage systems and I/O libraries, planning for purchases, comparing performance of application clusters or workloads across platforms, and improving the performance of parallel I/O libraries and applications. The analytics and benchmark generation software, and example benchmarks, will be publicly released. This project uses two methods to construct personalized I/O benchmarks. First, the project is making existing applications self-benchmarking across all of their runs, by providing analytics and visualization facilities to convey to stakeholders the information already automatically captured by lightweight I/O profilers such as Darshan during each run. Second, the project is creating platform-customized benchmark suites that represent the mix of application-level workloads observed on a given platform. To accomplish this, the project is clustering observed production jobs based on their I/O behavior and using both new and existing I/O kernel generation techniques to generate a compact benchmark for each cluster. The resulting benchmark suite will advance the state of the art by serving as a proxy for real-world, platform-specific production I/O workloads, and by providing previously unavailable insight into how prevalent those workloads are at a given facility.