Collaborative Research: Personalized Benchmarks for High Performance Computing Applications

随着高性能计算应用程序针对越来越大的问题，数据输入和输出(I/O)占用了越来越多的运行时间。用户、软件开发人员和平台管理员经常发现很难理解应用程序的I/O代码在做什么，为什么它很慢，如何改进它，或者它在不同的平台上执行得有多好。I/O基准有助于解决这个问题，但它们的生产成本很高，因此不适用于大多数应用程序。该项目利用已在高性能计算平台上监控应用程序行为的现有轻量级I/O分析器，为所有应用程序提供用户友好的个性化I/O基准测试。由此产生的个性化基准将帮助研究人员、开发人员和购买者评估潜在的新存储系统体系结构，评估现有或新版本的存储系统和I/O库，规划购买，比较应用程序群集或跨平台工作负载的性能，以及提高并行I/O库和应用程序的性能。分析和基准生成软件以及示例基准将公开发布。该项目使用两种方法来构建个性化I/O基准。首先，该项目正在使现有应用程序在其所有运行过程中进行自我基准测试，方法是提供分析和可视化设施，向利益相关者传达已由轻量级I/O分析器(如Darhan)在每次运行期间自动捕获的信息。其次，该项目正在创建平台定制的基准测试套件，这些套件代表在给定平台上观察到的应用级工作负载的混合。为此，该项目基于I/O行为对观察到的生产作业进行集群，并使用新的和现有的I/O内核生成技术为每个集群生成紧凑基准。最终的基准测试套件将作为特定于平台的真实生产I/O工作负载的代理，并提供以前无法获得的有关这些工作负载在给定设施中的普遍程度的洞察，从而推动最先进的技术。

As high-performance computing applications target ever-larger problems, data input and output (I/O) takes up more and more run time. Users, software developers, and platform administrators often find it difficult to understand what an application's I/O code is doing, why it is slow, how it might be improved, or how well it would perform on a different platform. I/O benchmarks help address this problem, but they are expensive to produce and thus are not available for most applications. This project is providing user-friendly personalized I/O benchmarks for all applications, by leveraging existing lightweight I/O profilers that already monitor the behavior of applications on high-performance computing platforms. The resulting personalized benchmarks will help researchers, developers, and purchasers in evaluating potential new storage system architectures, evaluating existing or new versions of storage systems and I/O libraries, planning for purchases, comparing performance of application clusters or workloads across platforms, and improving the performance of parallel I/O libraries and applications. The analytics and benchmark generation software, and example benchmarks, will be publicly released. This project uses two methods to construct personalized I/O benchmarks. First, the project is making existing applications self-benchmarking across all of their runs, by providing analytics and visualization facilities to convey to stakeholders the information already automatically captured by lightweight I/O profilers such as Darshan during each run. Second, the project is creating platform-customized benchmark suites that represent the mix of application-level workloads observed on a given platform. To accomplish this, the project is clustering observed production jobs based on their I/O behavior and using both new and existing I/O kernel generation techniques to generate a compact benchmark for each cluster. The resulting benchmark suite will advance the state of the art by serving as a proxy for real-world, platform-specific production I/O workloads, and by providing previously unavailable insight into how prevalent those workloads are at a given facility.