CAREER: Self-tuning Parallel Software and Systems

机器学习(ML)方法的最新进展正在推动许多学科的科学发现。这为并行计算社区提供了一个独特的机会，可以消除性能工程循环中的人为和相关猜测，转而使用数据驱动的ML模型来进行性能建模、预测和调优。对并行系统的软件性能和运行效率的数据分析可以用来确定性能异常及其根本原因。这可以改变优化并行软件性能和并行系统运行效率的过程。通过使用基于机器学习的数据驱动统计建模，可以最大限度地减少过程中人为错误的影响，并行软件和系统可以真正实现自我调整。这项工作利用并促进了越来越多关于ML for Systems的工作，并为极端规模的并行软件和系统带来了好处。该项目还吸引了高中生，对本科生和研究生进行并行计算培训，并为他们在HPC的职业生涯做好准备，以解决HPC行业和国家实验室严重缺乏计算机和计算科学家的问题。该项目正在应用统计和ML算法来分析性能数据，并使用训练后的模型和见解来实现并行软件和系统的性能自我调整。这项工作正在开发一种全面的方法来完成以下任务：(1)分析随着时间的推移收集的大量软件和系统数据，(2)应用机器学习来建模应用程序和系统行为，以及(3)使用这些模型来指导影响未来执行的应用程序、运行时和系统优化决策。这种数据驱动自调优的整体方法可以显著提高并行软件的性能和可移植性，以及HPC和数据中心系统的运营效率，即使代码和系统不断演变也是如此。个别工作的更好表现会导致更快的科学结果和更高的工作吞吐量。这项工作在三个关键领域取得了进展。首先，开发基于ML的机制来对并行软件的性能进行建模，并使用这些模型通过选择高性能配置来自动优化其性能。第二，开发用于分析并行系统的大规模纵向监测数据的自动化方法，并开发使用训练的ML模型来自动调整并行系统运行的机制。最后，当并行代码被移植到新的或未来的体系结构时，前两个推力可以用于通过使用迁移学习等技术来自动调整并行代码的性能。该项目正在开发一套技术和框架，以分析在不同级别(工作、系统和设施)收集的与绩效相关的数据，并为优化各种与运营效率相关的指标做出决策。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Recent advances in machine learning (ML) approaches are driving scientific discovery across many disciplines. This presents a unique opportunity in the parallel computing community to remove the human and associated guesswork in the performance engineering loop, and instead, use data-driven ML models for performance modeling, forecasting and tuning. Analytics of data about software performance and operational efficiency of the parallel systems can be used to identify performance anomalies and their root causes. This can transform the process of optimizing the performance of parallel software and operational efficiency of parallel systems. By using data-driven statistical modeling based on machine learning, the impact of human errors in the process can be minimized, and parallel software and systems can become truly self-tuning. This work is leveraging and contributing to the growing body of work on ML for Systems, and brings its benefits to extreme-scale parallel software and systems. The project is also engaging high school students, training undergraduate and graduate students in parallel computing and preparing them for a career in HPC to address a significant shortage of computer and computational scientists in HPC, both in the industry and national laboratories. The project is applying statistical and ML algorithms to analyze performance data, and using the trained models and insights to enable the self-tuning of performance of parallel software and systems. This work is developing a holistic methodology for accomplishing the following tasks: (1) analyze large volumes of software and system data collected over time, (2) apply machine learning to model application and system behavior, and (3) use these models to guide application, runtime and system optimization decisions that impact future executions. This holistic approach of data-driven self-tuning can significantly improve the performance and portability of parallel software, and operational efficiency of HPC and data center systems even as codes and systems evolve. Better performance of individual jobs leads to faster science results and increased job throughput. This work is making advances in three key areas. First, development of ML-based mechanisms to model the performance of parallel software and use of such models to automatically optimize their performance by selecting high-performance configurations. Second, the development of automated methods to analyze large-scale longitudinal monitoring data for analysis of parallel systems, and develop mechanisms to use trained ML models to automatically tune the operation of parallel systems. And finally, the first two thrusts can be used to automatically tune the performance of parallel codes as they are ported to new or future architectures by using techniques such as transfer learning. This project is leading to the development of a suite of techniques and frameworks to analyze performance-related data being gathered at different levels (job, system and facility) and to make decisions for optimizing various operational efficiency related metrics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.