OAC Core: Transpass: Transpiling Parallel Task Graph Programming Models for Scientific Software

任务图编程模型(TGPM)是许多机器学习系统中的重要组成部分，因为它允许自上而下地优化并行性，从而控制宏观性能。由于每个TGPM在计算特定的机器学习工作负载时表现出不同的性能约束，因此没有一个单独的TGPM可以保证所有TGPM的最佳性能。因此，帮助研究人员在不同的TGPM之间转换机器学习任务图对于提高当今许多以机器学习为中心的科学软件的加速硬件的利用率非常有利。不幸的是，没有足够的系统来简化转换过程，导致了大量的周转时间，这阻碍了研究人员在不同的计算平台上快速优化机器学习算法的性能。该项目创建了一个新的开源系统，1)实现了不同TGPM之间机器学习任务图的自动转换，2)为不同的研究人员群体建立了一个开放的平台，为程序转换研究和教育做出贡献。这个项目建立了一个新的基于任务图的编程环境Transpass，它可以简化不同TGPM之间的程序转换，特别关注机器学习任务图。TRANSPASS引入了一种新的中间表示，称为控制任务流图，用于表示具有图内控制流的单个任务图中的端到端并行性。在这种中间表示的基础上，Transpass引入了一个新的源代码到源代码编译器(“Transpeller”)，通过转换后的控制任务流图实现不同TGPM之间程序的多方向转换。为了支持转换图的高效执行，Transpass引入了一个新的基于学习的运行时，它可以在给定的软件和硬件环境中学习复杂的调度参数。该项目的技术贡献横跨多学科研究社区，包括高性能计算、程序转换和机器学习。TRANSPASS将是开源的，以促进对许多编程模型的通用性，并在计算社区中促进广泛的贡献。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Task graph programming model (TGPM) is an essential component in many machine learning systems because it allows top-down optimization of parallelism that governs macroscale performance. Due to the distinct performance constraints exhibited by each TGPM when computing a specific machine learning workload, no single TGPM can guarantee optimal performance for all. Assisting researchers in transforming machine learning task graphs between different TGPMs is thus extremely beneficial for advancing the utilization of acceleration hardware for many of today’s machine learning-centric scientific software. Unfortunately, insufficient systems to streamline the transformation process has resulted in a significant turnaround time that prevents researchers from quickly optimizing the performance of machine learning algorithms on different computing platforms. This project creates a novel open-source system that 1) enables an automatic transformation of machine learning task graphs between different TGPMs and 2) establishes an open platform for a diverse group of researchers to contribute to program transformation research and education. This project establishes Transpass, a novel task graph-based programming environment to streamline the transformation of a program between different TGPMs with a specific focus on machine learning task graphs. Transpass introduces a new intermediate representation called control taskflow graph to represent end-to-end parallelism in a single task graph with in-graph control flow. Atop this intermediate representation, Transpass introduces a new source-to-source compiler (“transpiler”) to enable multidirectional transformations of programs between different TGPMs through transformed control taskflow graph. To support efficient execution of transformed graphs, Transpass introduces a new learning-based runtime that learns complex scheduling parameters in the given software and hardware environment. Technical contributions of this project span a multidisciplinary research community, including high-performance computing, program transformation, and machine learning. Transpass will be open-source to facilitate generalizability to many programming models and promote widespread contributions in the computing community.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.