AF: Medium: Collaborative Research: Beyond Sparsity: Refined Measures of Complexity for Linear Algebra

AF：媒介：协作研究：超越稀疏性：线性代数复杂性的精确度量

• 批准号: 1763481
• 负责人: Atri Rudra
• 金额: $ 46.94万
• 依托单位: SUNY at Buffalo
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-03-15 至 2023-02-28
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1763481&HistoricalAwards=false
• 关键词: AF; Medium; Collaborative; Research; Beyond

Modern data science applications exploit structure in real life data using machine learning (including deep learning) algorithms. At the core of most of these systems are algorithms for a branch of mathematics called linear algebra. In particular, a large portion of these algorithms utilize the fact that real life data has properties that can be captured using certain parsimonious linear algebraic structures. This project studies new, more powerful linear algebraic structures and algorithms that exploit these new structures. Given the fundamental importance of these algorithms, ideas generated from this project are expected to be implemented in widely deployed machine learning systems. The outreach component of this project involves (1) a technical workshop for researchers from diverse areas and (2) outreach events for K-12 students.A variety of problems in modern data science have been successfully characterized using a width. For example, one of the most common widths, the rank of a matrix, has a near-ubiquitous use across many applications. This project significantly expands the understanding of several recently proposed widths and extracts their full potential for positive practical outcomes. Furthermore, it contributes to the recently growing work on beyond worst-case analysis in linear algebra, machine learning and coding theory.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

现代数据科学应用使用机器学习（包括深度学习）算法来利用真实的生活数据中的结构。这些系统的核心是数学中一个叫做线性代数的分支的算法。特别地，这些算法的大部分利用了这样的事实，即真实的生活数据具有可以使用某些简约线性代数结构来捕获的属性。这个项目研究新的，更强大的线性代数结构和算法，利用这些新的结构。鉴于这些算法的重要性，从这个项目中产生的想法有望在广泛部署的机器学习系统中实现。该项目的外展部分包括（1）为来自不同领域的研究人员举办的技术研讨会和（2）为K-12学生举办的外展活动。例如，最常见的宽度之一，矩阵的秩，在许多应用中几乎无处不在。该项目大大扩展了对最近提出的几种宽度的理解，并提取了它们的全部潜力，以取得积极的实际成果。此外，该奖项还为线性代数、机器学习和编码理论中的最坏情况分析做出了贡献。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models

• 发表时间: 2021-11
• 期刊: ArXiv
• 作者: Beidi Chen;Tri Dao;Kaizhao Liang;Jiaming Yang;Zhao Song;A. Rudra;C. Ré

Simple Hardware-Efficient Long Convolutions for Sequence Modeling

• DOI: 10.48550/arxiv.2302.06646
• 发表时间: 2023-02
• 期刊: ArXiv
• 作者: Daniel Y. Fu;Elliot L. Epstein;Eric N. D. Nguyen;A. Thomas;Michael Zhang;Tri Dao;A. Rudra;Christopher Ré

Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers

• 发表时间: 2021-10
• 作者: Albert Gu;Isys Johnson;Karan Goel;Khaled Kamal Saab;Tri Dao;A. Rudra;Christopher R'e

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

• 发表时间: 2022-05
• 期刊: ArXiv
• 作者: Tri Dao;Daniel Y. Fu;Stefano Ermon;A. Rudra;Christopher R'e

Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps

• 发表时间: 2020-04
• 期刊: ArXiv
• 作者: Tri Dao;N. Sohoni;Albert Gu;Matthew Eichhorn;Amit Blonder;Megan Leszczynski;A. Rudra;Christopher Ré