Collaborative Research: SHF: Small: Rethinking Performance Variation for Emerging Applications - An Application-centric and Cross-layer Approach

高性能计算（HPC）正在迅速发展，到2021年将达到无与伦比的exaflops水平，届时第一个exascale系统将为科学生产做好准备。尽管在不允许其他用户访问系统的维护窗口期间通过简单的基准测试获得了峰值性能，但由于存储和网络上的应用程序内或应用程序间干扰，应用程序通常会遭受性能变化。其结果是系统利用率低，应用程序的洞察时间延长。为了应对这一挑战，该项目旨在开发内存和输入/输出（I/O）的新方法，可以显着减少大型科学应用程序的性能变化。该项目提供综合研究和教育活动，以培养HPC领域的下一代计算机研究人员和工程师，特别是那些来自代表性不足的群体，以加强美国在计算科学和工程方面的竞争力。该项目旨在解决HPC系统上的性能变化问题，使用一种新的以应用程序为中心的方法在整个系统堆栈。为了解决日益增加的资源竞争，一个选择性的提示共享方案的设计，以减少整体性能的变化，和集群分区技术的发展，以调节提示共享的规模。此外，一个反馈机制被纳入调整提示流量根据性能变化的减少程度。基于存储器访问相似性，共享高相似性的存储器页面或工作节点被分组在一起以优化存储器系统性能。此外，基于规则的I/O重新路由方案，其中I/O流量不仅基于干扰简档而且基于下游数据分析的要求来重新路由。特别是，错误有界的粗化技术，通过调整HPC应用程序的保真度的性能变化的反应进行了探讨。该项目中的综合研究活动将显著提高对大型计算科学和工程应用的性能变化管理的理解和方法。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

High-performance computing (HPC) is moving rapidly to the unparalleled level of exaflops in 2021, when the first exascale systems will be ready for science production. Despite the peak performance obtained by simplistic benchmarks during the maintenance window when no other users are allowed to access the system, applications routinely suffer from performance variations as a result of intra- or inter-application interference over storage and network. The consequence is the low system utilization and prolonged time to insights for applications. To address this challenge, this project aims to develop new methods in memory and input/output (I/O) that can significantly reduce the performance variation for large scientific applications. This project provides integrated research and education activities to nurture next-generation computer researchers and engineers in the area of HPC, particularly for those from under-represented groups, to strengthen the U.S. competitiveness in computational science and engineering. This project aims to address the performance variation issue on HPC systems using a novel application-centric approach across the system stack. To address increasing resource contention, a selective hint-sharing scheme is designed to reduce the overall performance variation, and a cluster-partition technique is developed to regulate the scale of hint sharing. In addition, a feedback mechanism is incorporated to adjust the hint traffic according to the degree of performance-variation reduction. Based upon memory-access similarity, memory pages or work nodes sharing high similarity are grouped together to optimize the memory-system performance. Furthermore, a rule-based I/O re-routing scheme, where I/O traffic is re-routed based upon not only the interference profile, but also the requirements of downstream data analytics. In particular, an error-bounded coarsening technique that reacts to performance variation by adjusting the fidelity of an HPC application is explored. The integrated research activities in this project will significantly improve the understanding and methods in managing performance variations for large computational science and engineering applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.