Collaborative Research: CIF: Medium: Analysis and Geometry of Neural Dynamical Systems

合作研究:CIF:媒介:神经动力系统的分析和几何

基本信息

  • 批准号:
    2106377
  • 负责人:
  • 金额:
    $ 52.95万
  • 依托单位:
  • 依托单位国家:
    美国
  • 项目类别:
    Continuing Grant
  • 财政年份:
    2021
  • 资助国家:
    美国
  • 起止时间:
    2021-06-01 至 2024-05-31
  • 项目状态:
    已结题

项目摘要

The complexity of modern neural nets, with their millions of parameters and unprecedented computational demands, has been a major hurdle for the conventional approaches which had been successfully applied in machine learning over the past decades. This project aims to develop new mathematical and computational foundations for the analysis and design of these systems through a radically new conceptualization of their architectures as continuous dynamical systems. The key pillar of this framework is the idealization of depth as a continuum of layers and width as a continuum of neurons. Infinitesimal abstractions of this type have successfully unlocked many disciplines throughout the twentieth century, including probability, optimization, control, and many more. This collaborative project involving UIUC and MIT will push the boundaries of the theory and practice of deep learning, while sparking sustained interactions between the communities of electrical engineering, mathematics, statistics, and theoretical computer science. The project will also have broad impacts through a deliberate approach to education and training. The education and outreach activities will include research opportunities for undergraduate students at both institutions, as well as an exchange program to foster the collaboration and exchange of ideas. This project on Analysis and Geometry of Neural Dynamical Systems is developing the mathematical foundations of deep learning by synthesizing tools from probability, statistics, dynamical systems, geometric analysis, partial differential equations, and optimal transport. The research program is articulated around three major directions: (1) continuous models of neural dynamical systems; (2) discretization schemes; and (3) algorithms. The first direction is focusing on characterizing the tradeoffs between the expressive power and complexity of idealized infinitely wide and deep neural nets. The second direction builds on these continuous abstractions to develop, from first principles, mathematically rigorous and practically implementable techniques for analyzing large but finite neural nets. The third direction emphasizes algorithmic and computational aspects, such as the computational complexity of numerical methods, stability, and implicit regularization, using a novel synthesis of analytic and geometric methods developed as part of the project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
现代神经网络的复杂性,以及数以百万计的参数和前所未有的计算需求,已经成为过去几十年来成功应用于机器学习的传统方法的主要障碍。该项目旨在通过将其架构作为连续动力系统的全新概念,为这些系统的分析和设计开发新的数学和计算基础。该框架的关键支柱是将深度理想化为层的连续体,将宽度理想化为神经元的连续体。这种类型的无限小抽象成功地开启了整个20世纪的许多学科,包括概率、优化、控制等等。这个由UIUC和MIT参与的合作项目将推动深度学习理论和实践的边界,同时激发电气工程、数学、统计学和理论计算机科学社区之间的持续互动。该项目还将通过审慎的教育和培训办法产生广泛的影响。教育和推广活动将包括为两所大学的本科生提供研究机会,以及促进合作和思想交流的交流项目。这个关于神经动力系统的分析和几何的项目是通过综合概率、统计、动力系统、几何分析、偏微分方程和最优运输等工具来发展深度学习的数学基础。研究方向围绕三个主要方向:(1)神经动力系统的连续模型;(2)离散化方案;(3)算法。第一个方向侧重于描述理想化的无限宽和深度神经网络的表达能力和复杂性之间的权衡。第二个方向是建立在这些连续抽象的基础上,从第一原理出发,发展数学上严谨和实际可实现的技术,用于分析大型但有限的神经网络。第三个方向强调算法和计算方面,例如数值方法的计算复杂性,稳定性和隐式正则化,使用作为项目一部分开发的解析和几何方法的新颖综合。该奖项反映了美国国家科学基金会的法定使命,并通过使用基金会的知识价值和更广泛的影响审查标准进行评估,被认为值得支持。

项目成果

期刊论文数量(5)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
GULP: a prediction-based metric between representation
GULP:表示之间基于预测的度量
An Algorithmic Solution to the Blotto Game using Multi-marginal Couplings
Gaussian discrepancy: A probabilistic relaxation of vector balancing
高斯差异:矢量平衡的概率松弛
  • DOI:
    10.1016/j.dam.2022.08.007
  • 发表时间:
    2022
  • 期刊:
  • 影响因子:
    1.1
  • 作者:
    Chewi, Sinho;Gerber, Patrik;Rigollet, Philippe;Turner, Paxton
  • 通讯作者:
    Turner, Paxton
Bures-Wasserstein Barycenters and Low-Rank Matrix Recovery
  • DOI:
  • 发表时间:
    2022-10
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Tyler Maunu;Thibaut Le Gouic;P. Rigollet
  • 通讯作者:
    Tyler Maunu;Thibaut Le Gouic;P. Rigollet
Variational inference via Wasserstein gradient flows
  • DOI:
    10.48550/arxiv.2205.15902
  • 发表时间:
    2022-05
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Marc Lambert;Sinho Chewi;F. Bach;S. Bonnabel;P. Rigollet
  • 通讯作者:
    Marc Lambert;Sinho Chewi;F. Bach;S. Bonnabel;P. Rigollet
{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Philippe Rigollet其他文献

THÈSE DE DOCTORAT ÈS MATHÉMATIQUES
数学博士论文
  • DOI:
  • 发表时间:
    2016
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Bodhisattva Sen;Richard Nickl;Vladimir Koltchinskii;Philippe Rigollet;Arnak S. Dalalyan
  • 通讯作者:
    Arnak S. Dalalyan

Philippe Rigollet的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Philippe Rigollet', 18)}}的其他基金

Collaborative Research: Statistical Estimation with Algebraic Structure
合作研究:代数结构的统计估计
  • 批准号:
    1712596
  • 财政年份:
    2017
  • 资助金额:
    $ 52.95万
  • 项目类别:
    Continuing Grant
Statistical and Computational Tradeoffs in High Dimensional Learning
高维学习中的统计和计算权衡
  • 批准号:
    1541100
  • 财政年份:
    2015
  • 资助金额:
    $ 52.95万
  • 项目类别:
    Continuing Grant
CAREER: Large Scale Stochastic Optimization and Statistics
职业:大规模随机优化和统计
  • 批准号:
    1541099
  • 财政年份:
    2015
  • 资助金额:
    $ 52.95万
  • 项目类别:
    Continuing Grant
Statistical and Computational Tradeoffs in High Dimensional Learning
高维学习中的统计和计算权衡
  • 批准号:
    1317308
  • 财政年份:
    2013
  • 资助金额:
    $ 52.95万
  • 项目类别:
    Continuing Grant
CAREER: Large Scale Stochastic Optimization and Statistics
职业:大规模随机优化和统计
  • 批准号:
    1053987
  • 财政年份:
    2011
  • 资助金额:
    $ 52.95万
  • 项目类别:
    Continuing Grant
Optimal Sequential Allocation in Dynamic Environments
动态环境中的最优顺序分配
  • 批准号:
    0906424
  • 财政年份:
    2009
  • 资助金额:
    $ 52.95万
  • 项目类别:
    Standard Grant

相似国自然基金

Research on Quantum Field Theory without a Lagrangian Description
  • 批准号:
    24ZR1403900
  • 批准年份:
    2024
  • 资助金额:
    0.0 万元
  • 项目类别:
    省市级项目
Cell Research
  • 批准号:
    31224802
  • 批准年份:
    2012
  • 资助金额:
    24.0 万元
  • 项目类别:
    专项基金项目
Cell Research
  • 批准号:
    31024804
  • 批准年份:
    2010
  • 资助金额:
    24.0 万元
  • 项目类别:
    专项基金项目
Cell Research (细胞研究)
  • 批准号:
    30824808
  • 批准年份:
    2008
  • 资助金额:
    24.0 万元
  • 项目类别:
    专项基金项目
Research on the Rapid Growth Mechanism of KDP Crystal
  • 批准号:
    10774081
  • 批准年份:
    2007
  • 资助金额:
    45.0 万元
  • 项目类别:
    面上项目

相似海外基金

Collaborative Research: CIF: Medium: Snapshot Computational Imaging with Metaoptics
合作研究:CIF:Medium:Metaoptics 快照计算成像
  • 批准号:
    2403122
  • 财政年份:
    2024
  • 资助金额:
    $ 52.95万
  • 项目类别:
    Standard Grant
Collaborative Research: CIF-Medium: Privacy-preserving Machine Learning on Graphs
合作研究:CIF-Medium:图上的隐私保护机器学习
  • 批准号:
    2402815
  • 财政年份:
    2024
  • 资助金额:
    $ 52.95万
  • 项目类别:
    Standard Grant
Collaborative Research: CIF: Small: Mathematical and Algorithmic Foundations of Multi-Task Learning
协作研究:CIF:小型:多任务学习的数学和算法基础
  • 批准号:
    2343599
  • 财政年份:
    2024
  • 资助金额:
    $ 52.95万
  • 项目类别:
    Standard Grant
Collaborative Research: CIF: Small: Mathematical and Algorithmic Foundations of Multi-Task Learning
协作研究:CIF:小型:多任务学习的数学和算法基础
  • 批准号:
    2343600
  • 财政年份:
    2024
  • 资助金额:
    $ 52.95万
  • 项目类别:
    Standard Grant
Collaborative Research: CIF-Medium: Privacy-preserving Machine Learning on Graphs
合作研究:CIF-Medium:图上的隐私保护机器学习
  • 批准号:
    2402817
  • 财政年份:
    2024
  • 资助金额:
    $ 52.95万
  • 项目类别:
    Standard Grant
Collaborative Research: NSF-AoF: CIF: Small: AI-assisted Waveform and Beamforming Design for Integrated Sensing and Communication
合作研究:NSF-AoF:CIF:小型:用于集成传感和通信的人工智能辅助波形和波束成形设计
  • 批准号:
    2326622
  • 财政年份:
    2024
  • 资助金额:
    $ 52.95万
  • 项目类别:
    Standard Grant
Collaborative Research: CIF-Medium: Privacy-preserving Machine Learning on Graphs
合作研究:CIF-Medium:图上的隐私保护机器学习
  • 批准号:
    2402816
  • 财政年份:
    2024
  • 资助金额:
    $ 52.95万
  • 项目类别:
    Standard Grant
Collaborative Research: CIF: Medium: Snapshot Computational Imaging with Metaoptics
合作研究:CIF:Medium:Metaoptics 快照计算成像
  • 批准号:
    2403123
  • 财政年份:
    2024
  • 资助金额:
    $ 52.95万
  • 项目类别:
    Standard Grant
Collaborative Research: NSF-AoF: CIF: Small: AI-assisted Waveform and Beamforming Design for Integrated Sensing and Communication
合作研究:NSF-AoF:CIF:小型:用于集成传感和通信的人工智能辅助波形和波束成形设计
  • 批准号:
    2326621
  • 财政年份:
    2024
  • 资助金额:
    $ 52.95万
  • 项目类别:
    Standard Grant
Collaborative Research: CIF: Small: Versatile Data Synchronization: Novel Codes and Algorithms for Practical Applications
合作研究:CIF:小型:多功能数据同步:实际应用的新颖代码和算法
  • 批准号:
    2312872
  • 财政年份:
    2023
  • 资助金额:
    $ 52.95万
  • 项目类别:
    Standard Grant
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了