CIF: Medium: Collaborative Research: Coded Computing for Large-Scale Machine Learning

CIF:媒介:协作研究:大规模机器学习的编码计算

基本信息

  • 批准号:
    1763657
  • 负责人:
  • 金额:
    $ 30万
  • 依托单位:
  • 依托单位国家:
    美国
  • 项目类别:
    Continuing Grant
  • 财政年份:
    2018
  • 资助国家:
    美国
  • 起止时间:
    2018-09-01 至 2023-08-31
  • 项目状态:
    已结题

项目摘要

Deep learning models are breaking new ground in data science tasks including image recognition, automatic translation and autonomous driving. This is achieved by neural networks that can be hundreds of layers deep and involve hundreds of millions of parameters. Training such large models requires distributed computations, very long training times and expensive hardware. This project studies coding theoretic techniques that can accelerate distributed machine learning and allow training with cheaper commodity hardware. Beyond the development of theoretical foundations, this project develops new algorithms for providing fault tolerance over unreliable cloud infrastructure that can significantly reduce the cost of large-scale machine learning. The research outcomes of the project will be broadly disseminated and integrated into education. The specific focus of this research program is on mitigating the bottlenecks of distributed machine learning. Currently, scaling benefits are limited because of two reasons: first, communication is typically the bottleneck and second, straggler effects limit performance. Both problems can be mitigated using coding theoretic methods. This work proposes "coded computing", a transformative framework that combines coding theory with distributed computing to inject computational redundancy in a novel coded form. This framework is then used to develop three research thrusts: a) Coding for Linear Algebraic Computations b) Coding for Iterative Computations and c) Coding for General Distributed Computations. Each of the thrusts operates on a different layer of a machine learning pipeline but all rely on coding theoretic tools and distributed information processing.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
深度学习模型正在图像识别、自动翻译和自动驾驶等数据科学任务中开辟新的天地。这是通过神经网络实现的,神经网络可以有数百层深,涉及数亿个参数。训练如此大型的模型需要分布式计算、非常长的训练时间和昂贵的硬件。这个项目研究编码理论技术,可以加速分布式机器学习,并允许使用更便宜的商用硬件进行培训。除了理论基础的发展,该项目还开发了新的算法,用于在不可靠的云基础设施上提供容错,可以显著降低大规模机器学习的成本。该项目的研究成果将广泛传播并纳入教育。这一研究计划的具体重点是缓解分布式机器学习的瓶颈。目前,扩展的好处是有限的,原因有两个:第一,通信通常是瓶颈;第二,掉队效应限制了性能。这两个问题都可以使用编码理论方法来缓解。这项工作提出了“编码计算”,这是一种将编码理论与分布式计算相结合的变革性框架,以一种新的编码形式注入计算冗余。然后,该框架被用于开发三个研究主题:a)线性代数计算的编码;b)迭代计算的编码;c)通用分布式计算的编码。每一项努力都在机器学习管道的不同层面上运行,但都依赖于编码理论工具和分布式信息处理。这一奖项反映了NSF的法定使命,并通过使用基金会的智力优势和更广泛的影响审查标准进行评估,被认为值得支持。

项目成果

期刊论文数量(13)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Numerically Stable Polynomially Coded Computing
E-Approximate Coded Matrix Multiplication is Nearly Twice as Efficient as Exact Multiplication
电子近似编码矩阵乘法的效率几乎是精确乘法的两倍
On the Optimal Recovery Threshold of Coded Matrix Multiplication
  • DOI:
    10.1109/tit.2019.2929328
  • 发表时间:
    2020-01-01
  • 期刊:
  • 影响因子:
    2.5
  • 作者:
    Dutta, Sanghamitra;Fahim, Mohammad;Grover, Pulkit
  • 通讯作者:
    Grover, Pulkit
Addressing Unreliability in Emerging Devices and Non-von Neumann Architectures Using Coded Computing
  • DOI:
    10.1109/jproc.2020.2986362
  • 发表时间:
    2020-08-01
  • 期刊:
  • 影响因子:
    20.6
  • 作者:
    Dutta, Sanghamitra;Jeong, Haewon;Grover, Pulkit
  • 通讯作者:
    Grover, Pulkit
ϵ -Approximate Coded Matrix Multiplication Is Nearly Twice as Efficient as Exact Multiplication
ϵ - 近似编码矩阵乘法的效率几乎是精确乘法的两倍
{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Viveck Cadambe其他文献

A signal model for forensic DNA mixtures
法医 DNA 混合物的信号模型

Viveck Cadambe的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Viveck Cadambe', 18)}}的其他基金

Collaborative Research: CIF: Small: Approximate Coded Computing - Fundamental Limits of Precision, Fault-Tolerance, and Privacy
协作研究:CIF:小型:近似编码计算 - 精度、容错性和隐私的基本限制
  • 批准号:
    2231706
  • 财政年份:
    2023
  • 资助金额:
    $ 30万
  • 项目类别:
    Standard Grant
CNS: Core: Small: Consistent, Geo-Distributed Data Stores on the Public Cloud Using Erasure Coding
CNS:核心:小型:使用纠删码在公共云上实现一致的地理分布式数据存储
  • 批准号:
    2211045
  • 财政年份:
    2022
  • 资助金额:
    $ 30万
  • 项目类别:
    Standard Grant
CAREER: An Information Theoretic Perspective of Consistent Distributed Storage Systems
职业:一致分布式存储系统的信息论视角
  • 批准号:
    1553248
  • 财政年份:
    2016
  • 资助金额:
    $ 30万
  • 项目类别:
    Continuing Grant
CRII: CIF: Towards a Systematic Interference Alignment Approach for Network Information Flow
CRII:CIF:迈向网络信息流的系统干扰对齐方法
  • 批准号:
    1464336
  • 财政年份:
    2015
  • 资助金额:
    $ 30万
  • 项目类别:
    Standard Grant

相似海外基金

Collaborative Research: CIF: Medium: Snapshot Computational Imaging with Metaoptics
合作研究:CIF:Medium:Metaoptics 快照计算成像
  • 批准号:
    2403122
  • 财政年份:
    2024
  • 资助金额:
    $ 30万
  • 项目类别:
    Standard Grant
Collaborative Research: CIF-Medium: Privacy-preserving Machine Learning on Graphs
合作研究:CIF-Medium:图上的隐私保护机器学习
  • 批准号:
    2402815
  • 财政年份:
    2024
  • 资助金额:
    $ 30万
  • 项目类别:
    Standard Grant
Collaborative Research: CIF-Medium: Privacy-preserving Machine Learning on Graphs
合作研究:CIF-Medium:图上的隐私保护机器学习
  • 批准号:
    2402817
  • 财政年份:
    2024
  • 资助金额:
    $ 30万
  • 项目类别:
    Standard Grant
Collaborative Research: CIF-Medium: Privacy-preserving Machine Learning on Graphs
合作研究:CIF-Medium:图上的隐私保护机器学习
  • 批准号:
    2402816
  • 财政年份:
    2024
  • 资助金额:
    $ 30万
  • 项目类别:
    Standard Grant
Collaborative Research: CIF: Medium: Snapshot Computational Imaging with Metaoptics
合作研究:CIF:Medium:Metaoptics 快照计算成像
  • 批准号:
    2403123
  • 财政年份:
    2024
  • 资助金额:
    $ 30万
  • 项目类别:
    Standard Grant
Collaborative Research: CIF: Medium: Fundamental Limits of Cache-aided Multi-user Private Function Retrieval
协作研究:CIF:中:缓存辅助多用户私有函数检索的基本限制
  • 批准号:
    2312229
  • 财政年份:
    2023
  • 资助金额:
    $ 30万
  • 项目类别:
    Continuing Grant
Collaborative Research: CIF: Medium: Statistical and Algorithmic Foundations of Distributionally Robust Policy Learning
合作研究:CIF:媒介:分布式稳健政策学习的统计和算法基础
  • 批准号:
    2312205
  • 财政年份:
    2023
  • 资助金额:
    $ 30万
  • 项目类别:
    Continuing Grant
Collaborative Research: CIF: Medium: Fundamental Limits of Privacy-Enhancing Technologies
合作研究:CIF:中:隐私增强技术的基本限制
  • 批准号:
    2312666
  • 财政年份:
    2023
  • 资助金额:
    $ 30万
  • 项目类别:
    Continuing Grant
Collaborative Research: CIF: Medium: Fundamental Limits of Cache-aided Multi-user Private Function Retrieval
协作研究:CIF:中:缓存辅助多用户私有函数检索的基本限制
  • 批准号:
    2312228
  • 财政年份:
    2023
  • 资助金额:
    $ 30万
  • 项目类别:
    Continuing Grant
Collaborative Research: CIF: Medium: Robust Learning over Graphs
协作研究:CIF:媒介:图上的鲁棒学习
  • 批准号:
    2312547
  • 财政年份:
    2023
  • 资助金额:
    $ 30万
  • 项目类别:
    Continuing Grant
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了