Collaborative Research: PPoSS: Large: A Comprehensive Framework for Efficient, Scalable, and Performance-Portable Tensor Applications

张量计算是科学计算和机器学习中许多大规模并行软件应用的基础，它们的有效实现对于它们所实现的重大进步至关重要。然而，随着摩尔定律的终结，两个关键挑战现在威胁着持续的进步：（1）随着晶体管成为有限资源，硬件定制对于维持改进的性能和能量效率至关重要，这需要算法架构协同设计方法的进步;（2）硬件架构的日益定制化和异构性加剧了应用程序已经令人生畏的挑战-开发人员的生产力和软件的性能可移植性。该项目汇集了具有算法/软件/硬件堆栈专业知识的研究人员，以应对这些挑战。该项目的影响包括：（1）通过算法架构协同设计，提高硬件架构的性能和能效;（2）提高软件应用程序开发人员的生产力，提高在各种目标平台上实现的性能，从而增强计算技术在科学和工业中的效益;（3）可扩展机器学习和科学计算应用的进展。该项目在沿着多个方向做出贡献：（1）编译器优化：强大的统一方法，用于自动优化密集张量计算，基于一系列目标计算平台上的多级超矩形平铺执行的非线性成本模型;（2）稀疏性的可扩展性：基于对数据的内在稀疏模式和相应的数据重用模式的分析，多级分块方法增强稀疏张量计算的可伸缩性;（3）算法架构协同设计：通过利用新的成本模型，开发用于密集和稀疏张量计算的加速器的硬件-软件协同设计的强大和通用的新方法;（4）正确性和准确性：开发技术以确保编译器转换和编译器/硬件设计空间探索的正确性和浮点准确性;（5）应用程序：利用开发的方法和工具来推进机器学习和科学计算的前沿应用，包括PDE求解器、量子多体仿真、机器学习中的张量网络和大规模图像分析。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Computations on tensors are fundamental to many large-scale parallel software applications in scientific computing and machine learning, and their efficient implementation has been crucial for the significant advances they have enabled. However, with the end of Moore’s Law, two critical challenges now threaten continued progress: (1) with transistors becoming a bounded resource, hardware customization is critical to sustaining improved performance and energy efficiency, requiring advances in algorithm-architecture co-design methodology; (2) increasing customization and heterogeneity of hardware architectures aggravates the already daunting challenges of application-developer productivity and performance-portability of software. This project brings together researchers with expertise spanning the algorithm/software/hardware stack to address these challenges. The project’s impacts include (1) improved performance and energy efficiency of hardware architectures through algorithm-architecture co-design; (2) increased developer productivity for software applications and the performance achieved on a variety of target platforms, which enhances the benefits of computing technology in science and industry; (3) advances in scalable machine-learning and scientific computing applications.The project makes contributions along multiple directions: (1) compiler optimization: powerful unified methodology for automated optimization of dense tensor computations, based on non-linear cost models for multi-level hyper-rectangular tiled execution on a range of target computing platforms; (2) scalability with sparsity: multi-level blocking methodology to enhance scalability of sparse-tensor computations, based on analysis of the intrinsic sparsity patterns of the data and the corresponding data-reuse patterns; (3) algorithm-architecture co-design: by leveraging new cost models, development of powerful and general new approaches for hardware-software co-design of accelerators for dense- and sparse-tensor computations; (4) correctness and accuracy: development of techniques to ensure correctness and floating-point accuracy with compiler transformations and compiler/hardware design-space exploration; (5) applications: use of the developed methodology and tools to advance cutting-edge applications in machine learning and scientific computing, including PDE solvers, quantum many-body simulation, tensor networks in machine learning, and large-scale image analysis.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.