SPX: Collaborative Research: Parallel Algorithm by Blocks - A Data-centric Compiler/runtime System for Productive Programming of Scalable Parallel Systems

对于应用程序开发人员来说，在可伸缩的并行和异构计算机系统上实现高生产率和高性能是一个具有挑战性的目标。基于消息传递接口（MPI）的并行编程是目前应用最广泛、最有效的可扩展并行应用程序开发方法；然而，与提供数据结构全局共享视图的编程模型相比，应用程序开发人员的生产力较低。相比之下，使用全局地址空间编程模型实现高性能和可伸缩性是具有挑战性的。该项目侧重于开发一个以数据为中心的编译器/运行时框架，“块并行算法”（PAbB），旨在为用户提供多种并行编程模型的综合积极属性，而不存在缺点。这个项目的主要新颖之处在于，它结合了用户洞察力、新的编译器优化和高级运行时支持，以实现对矩阵、张量和图进行操作的一类重要计算的生产力和性能。这项工作的主要广泛影响是，它可以显著降低来自各个领域的科学家的进入门槛，这些科学家希望在大规模并行系统上开发新的高性能应用程序，但目前发现现有的并行编程模型太难了。该项目汇集了一组具有软件堆栈专业知识的研究人员，为pab开发编译器工具和运行时系统，并演示其在计算科学和数据科学的许多应用程序中的使用。pab模型旨在与MPI协同工作；也就是说，pab程序可以在任何标准的MPI环境中执行，与其他本地MPI代码进行互操作。所建议的方法背后的关键思想是为用户提供目标数据结构的全局地址视图，只要求（在某些情况下是可选的）用户指定应该如何对数据进行分区，但让编译器/运行时处理从全局到本地的重新索引和节点间数据移动的繁琐方面。除了提高生产率之外，第二个重要的好处是启用了对动态负载平衡的系统支持。这种方法是在密集和稀疏矩阵、张量和图上的应用程序的背景下设计和演示的。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Achieving both high productivity and high performance on scalable parallel and heterogeneous computer systems is a challenging goal for application developers. Parallel programming with Message Passing Interface (MPI) is currently the most widely used and effective means of developing scalable parallel applications; however the productivity of application developers is lower than with programming models that offer a global shared view of data structures. In comparison, achieving high performance and scalability with global-address-space programming models is challenging. This project focuses on the development of a data-centric compiler/runtime framework, "Parallel Algorithms by Blocks" (PAbB), aimed at offering users the combined positive attributes of multiple parallel programming models without the disadvantages. The main novelty of this project is that it uses a combination of user insights, new compiler optimizations, and advanced runtime support to achieve both productivity and performance for an important class of computations that operate on matrices, tensors, and graphs. The main broader impact of the work is that it can significantly lower the barrier to entry for scientists from various domains who wish to develop new high-performance applications on large scale parallel systems, but presently find it too difficult with currently available parallel programming models. This project brings together a team of investigators, with expertise across the software stack, to develop compiler tools and runtime systems for PAbB and demonstrate its use across a number of applications from computational science and data science. The PAbB model is intended to work in concert with MPI; that is, PAbB programs can execute in any standard MPI environment, interoperating with other native MPI code. The key idea behind the proposed approach is to offer the user a global-address view of the targeted data structures, requiring only (optionally in some cases) that they specify how data should be partitioned, but have the compiler/runtime handle the tedious aspects of the global-to-local re-indexing and inter-node data movement. In addition to the productivity benefit, a second significant benefit is in enabling system support for dynamic load balancing. The approach is being designed and demonstrated in the context of applications operating on dense and sparse matrices and tensors, and graphs.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.