SPX: Collaborative Research: Parallel Algorithm by Blocks - A Data-centric Compiler/runtime System for Productive Programming of Scalable Parallel Systems

在可扩展的并行和异构计算机系统上实现高生产率和高性能是应用程序开发人员的一个具有挑战性的目标。使用消息传递接口（MPI）的并行编程是目前开发可扩展并行应用程序的最广泛使用和最有效的方法;然而，应用程序开发人员的生产力低于提供全局共享数据结构视图的编程模型。相比之下，使用全局地址空间编程模型实现高性能和可伸缩性是具有挑战性的。该项目的重点是开发一个以数据为中心的编译器/运行时框架，“并行算法块”（PAbB），旨在为用户提供多个并行编程模型的组合的积极属性，而没有缺点。该项目的主要新奇在于，它结合了用户洞察、新的编译器优化和高级运行时支持，以实现对矩阵、张量和图形进行运算的一类重要计算的生产力和性能。这项工作的主要影响是，它可以显着降低来自各个领域的科学家的进入门槛，这些科学家希望在大规模并行系统上开发新的高性能应用程序，但目前发现目前可用的并行编程模型太难了。该项目汇集了一个具有软件堆栈专业知识的研究人员团队，为PAbB开发编译器工具和运行时系统，并展示其在计算科学和数据科学的许多应用程序中的使用。PAbB模型旨在与MPI协同工作;也就是说，PAbB程序可以在任何标准MPI环境中执行，与其他本地MPI代码互操作。所提出的方法背后的关键思想是为用户提供目标数据结构的全局地址视图，仅需要（在某些情况下可选地）它们指定数据应如何分区，但编译器/运行时处理全局到本地重新索引和节点间数据移动的繁琐方面。除了生产力的好处之外，第二个重要的好处是支持动态负载平衡的系统支持。该方法正在密集和稀疏矩阵和张量和graphs.This奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的应用程序的背景下进行设计和演示。

Achieving both high productivity and high performance on scalable parallel and heterogeneous computer systems is a challenging goal for application developers. Parallel programming with Message Passing Interface (MPI) is currently the most widely used and effective means of developing scalable parallel applications; however the productivity of application developers is lower than with programming models that offer a global shared view of data structures. In comparison, achieving high performance and scalability with global-address-space programming models is challenging. This project focuses on the development of a data-centric compiler/runtime framework, "Parallel Algorithms by Blocks" (PAbB), aimed at offering users the combined positive attributes of multiple parallel programming models without the disadvantages. The main novelty of this project is that it uses a combination of user insights, new compiler optimizations, and advanced runtime support to achieve both productivity and performance for an important class of computations that operate on matrices, tensors, and graphs. The main broader impact of the work is that it can significantly lower the barrier to entry for scientists from various domains who wish to develop new high-performance applications on large scale parallel systems, but presently find it too difficult with currently available parallel programming models. This project brings together a team of investigators, with expertise across the software stack, to develop compiler tools and runtime systems for PAbB and demonstrate its use across a number of applications from computational science and data science. The PAbB model is intended to work in concert with MPI; that is, PAbB programs can execute in any standard MPI environment, interoperating with other native MPI code. The key idea behind the proposed approach is to offer the user a global-address view of the targeted data structures, requiring only (optionally in some cases) that they specify how data should be partitioned, but have the compiler/runtime handle the tedious aspects of the global-to-local re-indexing and inter-node data movement. In addition to the productivity benefit, a second significant benefit is in enabling system support for dynamic load balancing. The approach is being designed and demonstrated in the context of applications operating on dense and sparse matrices and tensors, and graphs.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.