Collaborative Research: PPoSS: Large: A Comprehensive Framework for Efficient, Scalable, and Performance-Portable Tensor Applications

张量的计算是科学计算和机器学习中许多大规模并行软件应用的基础，它们的有效实现对于它们所实现的重大进步至关重要。然而，随着摩尔定律的终结，现在有两个关键挑战威胁着持续的进步：(1)随着晶体管成为有限的资源，硬件定制对于保持性能和能源效率的提高至关重要，这需要算法架构协同设计方法的进步；(2)不断增加的定制和硬件架构的异构性加剧了应用程序开发人员生产力和软件性能可移植性的严峻挑战。该项目汇集了具有算法/软件/硬件堆栈专业知识的研究人员，以解决这些挑战。该项目的影响包括：(1)通过算法-架构协同设计提高了硬件架构的性能和能效；(2)提高了软件应用程序开发人员的生产力和在各种目标平台上实现的性能，从而增强了计算技术在科学和工业中的效益；(3)可扩展机器学习和科学计算应用的进展。该项目在多个方向上做出了贡献：(1)编译器优化：基于非线性成本模型，在一系列目标计算平台上执行多层次超矩形平铺执行，为密集张量计算的自动优化提供了强大的统一方法；(2)具有稀疏性的可扩展性：基于分析数据固有的稀疏性模式和相应的数据重用模式，采用多级阻塞方法增强稀疏张量计算的可扩展性；(3)算法-架构协同设计：利用新的成本模型，开发强大而通用的新方法，用于密集张量和稀疏张量计算的加速器的软硬件协同设计；(4)正确性和准确性：开发通过编译器转换和编译器/硬件设计空间探索来确保正确性和浮点精度的技术；(5)应用：利用开发的方法和工具推进机器学习和科学计算领域的前沿应用，包括PDE求解器、量子多体模拟、机器学习中的张量网络和大规模图像分析。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Computations on tensors are fundamental to many large-scale parallel software applications in scientific computing and machine learning, and their efficient implementation has been crucial for the significant advances they have enabled. However, with the end of Moore’s Law, two critical challenges now threaten continued progress: (1) with transistors becoming a bounded resource, hardware customization is critical to sustaining improved performance and energy efficiency, requiring advances in algorithm-architecture co-design methodology; (2) increasing customization and heterogeneity of hardware architectures aggravates the already daunting challenges of application-developer productivity and performance-portability of software. This project brings together researchers with expertise spanning the algorithm/software/hardware stack to address these challenges. The project’s impacts include (1) improved performance and energy efficiency of hardware architectures through algorithm-architecture co-design; (2) increased developer productivity for software applications and the performance achieved on a variety of target platforms, which enhances the benefits of computing technology in science and industry; (3) advances in scalable machine-learning and scientific computing applications.The project makes contributions along multiple directions: (1) compiler optimization: powerful unified methodology for automated optimization of dense tensor computations, based on non-linear cost models for multi-level hyper-rectangular tiled execution on a range of target computing platforms; (2) scalability with sparsity: multi-level blocking methodology to enhance scalability of sparse-tensor computations, based on analysis of the intrinsic sparsity patterns of the data and the corresponding data-reuse patterns; (3) algorithm-architecture co-design: by leveraging new cost models, development of powerful and general new approaches for hardware-software co-design of accelerators for dense- and sparse-tensor computations; (4) correctness and accuracy: development of techniques to ensure correctness and floating-point accuracy with compiler transformations and compiler/hardware design-space exploration; (5) applications: use of the developed methodology and tools to advance cutting-edge applications in machine learning and scientific computing, including PDE solvers, quantum many-body simulation, tensor networks in machine learning, and large-scale image analysis.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.