SPX: Collaborative Research: Parallel Algorithm by Blocks - A Data-centric Compiler/runtime System for Productive Programming of Scalable Parallel Systems

对于应用程序开发人员来说，在可伸缩的并行和异构计算机系统上实现高生产率和高性能是一个具有挑战性的目标。使用消息传递接口(MPI)的并行编程是目前开发可伸缩并行应用程序的最广泛和最有效的方法；然而，应用程序开发人员的生产率低于提供全局共享数据结构视图的编程模型。相比之下，使用全局地址空间编程模型实现高性能和可伸缩性是具有挑战性的。该项目致力于开发一个以数据为中心的编译器/运行时框架--“块并行算法”(PAbB)，旨在为用户提供多个并行编程模型的组合优点，而不是缺点。这个项目的主要新奇之处在于，它结合了用户洞察、新的编译器优化和高级运行时支持，以实现对矩阵、张量和图形操作的一类重要计算的生产力和性能。这项工作的主要更广泛的影响是，它可以显著降低来自不同领域的科学家的进入门槛，这些科学家希望在大规模并行系统上开发新的高性能应用程序，但目前发现使用现有的并行编程模型太难了。该项目汇集了一组拥有软件堆栈专业知识的调查人员，为PAbB开发编译器工具和运行时系统，并演示其在计算科学和数据科学的许多应用程序中的使用。PAbB模型旨在与MPI协同工作；也就是说，PAbB程序可以在任何标准MPI环境中执行，并与其他本机MPI代码互操作。提出的方法背后的关键思想是为用户提供目标数据结构的全局地址视图，只需要(在某些情况下可选)它们指定应该如何对数据进行分区，但让编译器/运行时处理从全局到本地的重建索引和节点间数据移动的繁琐方面。除了生产力方面的好处，第二个显著的好处是支持系统对动态负载平衡的支持。该方法是在对稠密和稀疏的矩阵和张量以及图形进行操作的应用程序的背景下设计和演示的。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Achieving both high productivity and high performance on scalable parallel and heterogeneous computer systems is a challenging goal for application developers. Parallel programming with Message Passing Interface (MPI) is currently the most widely used and effective means of developing scalable parallel applications; however the productivity of application developers is lower than with programming models that offer a global shared view of data structures. In comparison, achieving high performance and scalability with global-address-space programming models is challenging. This project focuses on the development of a data-centric compiler/runtime framework, "Parallel Algorithms by Blocks" (PAbB), aimed at offering users the combined positive attributes of multiple parallel programming models without the disadvantages. The main novelty of this project is that it uses a combination of user insights, new compiler optimizations, and advanced runtime support to achieve both productivity and performance for an important class of computations that operate on matrices, tensors, and graphs. The main broader impact of the work is that it can significantly lower the barrier to entry for scientists from various domains who wish to develop new high-performance applications on large scale parallel systems, but presently find it too difficult with currently available parallel programming models. This project brings together a team of investigators, with expertise across the software stack, to develop compiler tools and runtime systems for PAbB and demonstrate its use across a number of applications from computational science and data science. The PAbB model is intended to work in concert with MPI; that is, PAbB programs can execute in any standard MPI environment, interoperating with other native MPI code. The key idea behind the proposed approach is to offer the user a global-address view of the targeted data structures, requiring only (optionally in some cases) that they specify how data should be partitioned, but have the compiler/runtime handle the tedious aspects of the global-to-local re-indexing and inter-node data movement. In addition to the productivity benefit, a second significant benefit is in enabling system support for dynamic load balancing. The approach is being designed and demonstrated in the context of applications operating on dense and sparse matrices and tensors, and graphs.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.