CAREER: Next-Generation Infrastructure for Tensor Computations

职业：用于张量计算的下一代基础设施

• 批准号: 1942995
• 负责人: Edgar Solomonik
• 金额: $ 50万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-08-01 至 2025-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1942995&HistoricalAwards=false
• 关键词: CAREER; Next; Generation; Infrastructure; Tensor

Matrices and their higher-order generalization (tensors) provide a mathematical toolbox for expressing a large variety of algorithms. Consequently, linear algebra operations on dense matrices have served as the backbone of high-performance scientific computing applications. This research aims to translate this benefit to more complex problems, by improving software infrastructure and parallel performance of sparse matrix and tensor operations. The proposed methods will be applied to accelerate analysis of large graphs, approximation of multidimensional datasets by tensor decompositions, and simulation of quantum systems. By providing a high-level library for distributed sparse tensors, the research will improve the development productivity of scientists and engineers from disciplines including chemistry, physics, and bioinformatics. Deployment of tensor-based techniques on massively-parallel computing systems will enable simulations of larger scale and higher accuracy, making new innovations in computational science possible. Additionally, development of web-based educational modules for programming with tensors and understanding parallel performance will make the software and methods accessible to the broader scientific community.Tensor decompositions and tensor networks are fundamental techniques in approximation of multi-dimensional data and functions. The frontiers of tensor computations in quantum chemistry and data analysis involve methods that contract tensors of different order, size, and sparsity. Recent developments have led to provably efficient algorithms and software for contraction of a pair of dense tensors and multiplication of a pair of sparse matrices. However, in the context of sparse multi-tensor operations, opportunities for asymptotic cost improvements remain. In particular, there is a lack of software and rigorous algorithmic analysis for sparse matrix and tensor computations involving hyper-sparsity and output sparsity, as well as for all-at-once contraction of multiple tensors, which can be advantageous in the presence of sparsity. Further, at the software library level, open problems remain in leveraging layout persistence, reuse of mapping logic, and automated performance modeling. The project will address these gaps in the state-of-the-art of available computational infrastructure by developing new parallel algorithms and systems techniques for sparse multi-tensor contraction. These innovations will be integrated into the Cyclops library and studied in the context of applications in graph analysis, tensor decomposition, and tensor networks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

矩阵和它们的高阶推广（张量）提供了一个数学工具箱来表达各种各样的算法。因此，稠密矩阵上的线性代数运算已经成为高性能科学计算应用的支柱。本研究旨在通过改善软件基础设施和稀疏矩阵和张量运算的并行性能，将这种优势转化为更复杂的问题。所提出的方法将被应用于加速大型图的分析，张量分解的多维数据集的近似，以及量子系统的模拟。通过提供分布式稀疏张量的高级库，该研究将提高化学、物理和生物信息学等学科的科学家和工程师的开发生产力。在并行计算系统上部署基于张量的技术将使更大规模和更高精度的模拟成为可能，使计算科学的新创新成为可能。此外，开发基于网络的教育模块，用于使用张量编程和理解并行性能，将使更广泛的科学界能够使用该软件和方法。张量分解和张量网络是多维数据和函数近似的基本技术。量子化学和数据分析中张量计算的前沿涉及收缩不同阶、大小和稀疏度的张量的方法。最近的发展导致了可证明有效的算法和软件的一对密集张量的收缩和一对稀疏矩阵的乘法。然而，在稀疏多张量操作的背景下，渐近成本改进的机会仍然存在。特别是，缺乏软件和严格的算法分析，用于稀疏矩阵和张量计算，涉及超稀疏性和输出稀疏性，以及用于多个张量的一次性收缩，这在稀疏性存在的情况下可能是有利的。此外，在软件库级别，在利用布局持久性、映射逻辑的重用和自动化性能建模方面仍然存在开放问题。该项目将通过开发用于稀疏多张量收缩的新的并行算法和系统技术来解决现有计算基础设施的最新技术中的这些差距。这些创新将被集成到Cyclops库中，并在图分析、张量分解和张量网络的应用背景下进行研究。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Fast and Accurate Randomized Algorithms for Low-rank Tensor Decompositions

• 发表时间: 2021-04
• 期刊: ArXiv
• 作者: Linjian Ma;Edgar Solomonik

Distributed-memory tensor completion for generalized loss functions in python using new sparse tensor kernels

• DOI: 10.1016/j.jpdc.2022.07.005
• 发表时间: 2019-10
• 期刊: J. Parallel Distributed Comput.
• 作者: Navjot Singh;Zecheng Zhang;Xiaoxia Wu;Naijing Zhang;Siyuan Zhang;Edgar Solomonik

Parallel Minimum Spanning Forest Computation using Sparse Matrix Kernels

• DOI: 10.1137/1.9781611977141.7
• 发表时间: 2021-10
• 作者: Tim Baer;Raghavendra Kanakagiri;Edgar Solomonik

Cost-efficient Gaussian Tensor Network Embeddings for Tensor-structured Inputs

用于张量结构输入的经济高效的高斯张量网络嵌入

• 发表时间: 2023
• 期刊: Advances in neural information processing systems
• 作者: Ma, Linjian;Solomonik, Edgar
• 通讯作者: Solomonik, Edgar