EAGER: Workload Analysis of NSF Innovative HPC Program Resources

NSF创新的高性能计算(HPC)计划为在美国各地从事前沿科学和工程研究的数千名计算科学家提供关键的计算和数据分析能力。该计划支持的系统组合旨在技术多样化，反映出计算在研究和教育过程中的变化和日益增长的使用。此外，该计划补充和扩展了校园和其他地区性研究网络基础设施提供的能力。鉴于该计划中的系统在国家研究组合中发挥的重要和独特的作用，重要的是详细了解其过去和目前的工作量。该项目旨在对NSF创新高性能计算计划中的所有系统进行工作负载趋势分析。工作负载特征很重要，因为它是HPC系统优化性能调优的组成部分。此外，对创新的HPC计划中的所有系统进行集体检查不仅有助于更广泛的HPC生态系统的整体容量规划，还可以揭示计算科学研究本身的性质是如何随着时间的推移而演变的。工作负载分析将解决基本的使用和性能问题，例如：高吞吐量应用程序(大量松散耦合的串行、单节点和小型节点计数作业)和网关应用程序消耗了多少HPC程序资源，这种情况是否会随着时间的推移而变化？入门工作的特点是什么？有多少资源用于数据分析/数据密集型计算？整个生态系统的运行时超额认购能力是多少？系统之间的作业组合是否存在差异？如果存在，这对作业吞吐量有何影响？这项研究将利用XDMoD(XD Metrics On Demand)，它包含一个数据仓库，其中包含通过NSF创新HPC计划提供的所有资源的详细工作级别会计和绩效数据。此外，这项研究的结果不仅将为方案中系统的广大用户和维护员提供详细的操作和性能分析，而且还将用作对其他先进高性能计算机系统进行类似研究的模板。通过使用已经被世界各地的HPC中心广泛使用的Open XDMoD以及在各种会议上的陈述来促进知识的转移。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为是值得支持的。

The NSF Innovative High Performance Computing (HPC) program provides critical computational and data analytics capabilities to thousands of computational scientists conducting leading-edge science and engineering research across the U.S. The portfolio of systems supported by the program is intended to be technically diverse, reflecting changing and growing use of computation in both the research and education process. In addition, the program complements and extends the capabilities provided by campus and other regional research cyberinfrastructures. Given the important and unique role that the systems in the program play in the Nation's research portfolio, it is important to have a detailed understanding of its past and current workload. This project seeks to perform a workload trend analysis for all the systems in the NSF Innovative HPC program. Workload characterization is important because it is an integral part of optimal performance tuning for HPC Systems. In addition, examining all the systems within the Innovative HPC program collectively will not only aid holistic capacity planning for the wider HPC ecosystem, but also, reveal how the nature of computational science research itself is evolving over time. The workload analysis will address fundamental usage and performance questions such as: How much of the HPC program resources are consumed by high throughput applications (large numbers of loosely-coupled serial, single and small node count jobs) and gateway applications, and is this changing over time? What are the characteristics of gateway jobs? How much of the resources are used for data analytics/data intensive computing? What is the run-time over-subscription capacity for the entire ecosystem? Are there differences in the job mixes among the systems and if so, how does this impact job throughput? The study will leverage XDMoD (XD Metrics on Demand) which contains a data warehouse of detailed job level accounting and performance data for all the resources provided through the NSF Innovative HPC program. Moreover, the results of this study will not only provide detailed operational and performance analytics for the broad community of users and maintainers of the systems in the program, but will also be used as a template for similar studies carried out on other advanced HPC systems. This transfer of knowledge will be facilitated through the use of Open XDMoD, which is already in wide use by HPC centers worldwide, as well as presentations at conferences and meetings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.