CAREER: Neural Networks in the Practical Regime
职业:神经网络在实际应用中的应用
基本信息
- 批准号:2145630
- 负责人:
- 金额:$ 40.89万
- 依托单位:
- 依托单位国家:美国
- 项目类别:Continuing Grant
- 财政年份:2022
- 资助国家:美国
- 起止时间:2022-06-01 至 2027-05-31
- 项目状态:未结题
- 来源:
- 关键词:
项目摘要
This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2). Deep learning is the predominant approach in modern artificial intelligence that uses multi-layer artificial neural networks to infer complex relations from data. Despite the rapid adoption of deep learning in scientific and industrial applications, a theoretical basis to explain its success and methods to guard against its limits has yet to be established. This project addresses two critical gaps in the current theory of deep learning, namely the effects of working with specific types of data and the behavior of artificial neural networks in practical settings. This research is crucial to developing deep learning methods that are more efficient, reliable, safe, and broadly applicable. Some areas that would benefit directly are accelerating production pipelines, improving energy efficiency, and improving data handling in the increasing number of critical technologies that rely on deep learning. This project involves significant educational, community building, and outreach activities. The research will be directly integrated into interdisciplinary curricula and will generate research project topics for graduate students and capstone projects for undergraduate students. The project aims to prepare a diverse STEM workforce through long-term research and career mentoring for undergraduate and graduate students and postdocs, open seminars and discussion sessions with experts and enthusiasts at different career stages, internship opportunities, and dedicated training sessions.Artificial neural networks provide specific parametrizations to specific sets of candidate solutions to learning tasks, and parameter optimization procedures introduce specific preferences in the space of possible solutions. This project seeks to illuminate these complex relations in cases of practical interest that are not sufficiently well covered by existing mathematical theory, namely, where the networks have a moderate level of overparametrization in relation to the amount of training data. Importantly, it develops theories and methods that integrate and exploit the properties of the training data and parameter initialization. The research concerns the following three aims: (1) the function space description of moderately overparametrized networks, (2) the data-dependent description of the objective function and optimization, and (3) the explicit form of the bias of gradient descent in function space. The research program advances the state of the art by integrating the properties of the training data and addressing the optimization bias and model bias in interplay, which are challenges beyond the scope of existing methods. The project builds on preliminary work that blends applied mathematics and deep learning, in particular techniques connecting the geometry of parameter space, function space, and data space, as well as techniques based on information geometry, optimal transport, and algebraic statistics. The project will further develop important connections between geometry, probability, statistics, and machine learning and will offer unique opportunities for interdisciplinary research and education.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
该奖项全部或部分根据2021年美国救援计划法案(公法117-2)资助。深度学习是现代人工智能的主要方法,它使用多层人工神经网络从数据中推断复杂的关系。尽管深度学习在科学和工业应用中迅速普及,但解释其成功的理论基础和防止其局限性的方法尚未建立。该项目解决了当前深度学习理论中的两个关键空白,即处理特定类型数据的影响和人工神经网络在实际环境中的行为。这项研究对于开发更高效、可靠、安全和广泛适用的深度学习方法至关重要。一些直接受益的领域包括加速生产管道,提高能源效率,以及在越来越多的依赖深度学习的关键技术中改进数据处理。该项目涉及重要的教育、社区建设和外联活动。该研究将直接融入跨学科课程,并将为研究生和本科生的顶点项目产生研究项目主题。该项目旨在通过对本科生、研究生和博士后的长期研究和职业指导、与不同职业阶段的专家和爱好者的公开研讨会和讨论会、实习机会以及专门的培训课程,为多样化的STEM劳动力做好准备。人工神经网络为学习任务的特定候选解决方案集提供特定的参数化并且参数优化过程在可能的解的空间中引入特定的偏好。该项目旨在阐明这些复杂的关系,在实际利益的情况下,没有足够好地涵盖现有的数学理论,即,网络有一个适度的过度参数化的训练数据量。重要的是,它开发了整合和利用训练数据和参数初始化属性的理论和方法。本文的主要研究内容有三个:(1)适度过参数化网络的函数空间描述;(2)目标函数和优化的数据依赖描述;(3)梯度下降偏差在函数空间中的显式表示。该研究计划通过整合训练数据的属性并解决相互作用中的优化偏差和模型偏差来推进最新技术水平,这些偏差和偏差超出了现有方法的范围。该项目建立在融合应用数学和深度学习的初步工作的基础上,特别是连接参数空间、函数空间和数据空间几何的技术,以及基于信息几何、最优传输和代数统计的技术。该项目将进一步发展几何、概率、统计和机器学习之间的重要联系,并将为跨学科研究和教育提供独特的机会。该奖项反映了NSF的法定使命,并通过使用基金会的知识价值和更广泛的影响审查标准进行评估,被认为值得支持。
项目成果
期刊论文数量(8)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Characterizing the Spectrum of the NTK via a Power Series Expansion
- DOI:10.48550/arxiv.2211.07844
- 发表时间:2022-11
- 期刊:
- 影响因子:0
- 作者:Michael Murray;Hui Jin;Benjamin Bowman;Guido Montúfar
- 通讯作者:Michael Murray;Hui Jin;Benjamin Bowman;Guido Montúfar
Spectral Bias Outside the Training Set for Deep Networks in the Kernel Regime
- DOI:10.48550/arxiv.2206.02927
- 发表时间:2022-06
- 期刊:
- 影响因子:0
- 作者:Benjamin Bowman;Guido Montúfar
- 通讯作者:Benjamin Bowman;Guido Montúfar
Algebraic optimization of sequential decision problems
顺序决策问题的代数优化
- DOI:10.1016/j.jsc.2023.102241
- 发表时间:2024
- 期刊:
- 影响因子:0.7
- 作者:Dressler, Mareike;Garrote-López, Marina;Montúfar, Guido;Müller, Johannes;Rose, Kemal
- 通讯作者:Rose, Kemal
FoSR: First-order spectral rewiring for addressing oversquashing in GNNs
- DOI:10.48550/arxiv.2210.11790
- 发表时间:2022-10
- 期刊:
- 影响因子:0
- 作者:Kedar Karhadkar;P. Banerjee;Guido Montúfar
- 通讯作者:Kedar Karhadkar;P. Banerjee;Guido Montúfar
Critical points and convergence analysis of generative deep linear networks trained with Bures-Wasserstein loss
使用 Bures-Wasserstein 损失训练的生成深度线性网络的关键点和收敛分析
- DOI:
- 发表时间:2023
- 期刊:
- 影响因子:0
- 作者:Brechet, Pierre;Papagiannouli, Katerina;An, Jing;Montufar, Guido
- 通讯作者:Montufar, Guido
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Guido Montufar Cuartas其他文献
Guido Montufar Cuartas的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Guido Montufar Cuartas', 18)}}的其他基金
Collaborative Research: RI: Medium: MoDL: Occams Razor in Deep and Physical Learning
合作研究:RI:媒介:MoDL:深度学习和物理学习中的奥卡姆斯剃刀
- 批准号:
2212520 - 财政年份:2022
- 资助金额:
$ 40.89万 - 项目类别:
Standard Grant
相似国自然基金
Neural Process模型的多样化高保真技术研究
- 批准号:62306326
- 批准年份:2023
- 资助金额:30 万元
- 项目类别:青年科学基金项目
相似海外基金
CAREER: Rethinking Spiking Neural Networks from a Dynamical System Perspective
职业:从动态系统的角度重新思考尖峰神经网络
- 批准号:
2337646 - 财政年份:2024
- 资助金额:
$ 40.89万 - 项目类别:
Continuing Grant
CAREER: Reliable and Accelerated Deep Neural Networks via Co-Design of Hardware and Algorithms
职业:通过硬件和算法的协同设计实现可靠且加速的深度神经网络
- 批准号:
2340516 - 财政年份:2024
- 资助金额:
$ 40.89万 - 项目类别:
Continuing Grant
CAREER: Dynamic Distributed Learning in Spiking Neural Networks with Neural Architecture Search
职业:具有神经架构搜索的尖峰神经网络中的动态分布式学习
- 批准号:
2238227 - 财政年份:2023
- 资助金额:
$ 40.89万 - 项目类别:
Continuing Grant
CAREER: Deep Neural Networks That Can See Shape From Images: Models, Algorithms, and Applications
职业:可以从图像中看到形状的深度神经网络:模型、算法和应用
- 批准号:
2239977 - 财政年份:2023
- 资助金额:
$ 40.89万 - 项目类别:
Continuing Grant
CAREER: Towards Responsible Graph Neural Networks
职业:迈向负责任的图神经网络
- 批准号:
2238616 - 财政年份:2023
- 资助金额:
$ 40.89万 - 项目类别:
Standard Grant
CAREER: NeuralSAT: A Constraint-Solving Framework for Verifying Deep Neural Networks
职业:NeuralSAT:用于验证深度神经网络的约束求解框架
- 批准号:
2238133 - 财政年份:2023
- 资助金额:
$ 40.89万 - 项目类别:
Continuing Grant
CAREER: Intersubband neurons for ultrafast optical neural networks
职业:超快光学神经网络的子带间神经元
- 批准号:
2349259 - 财政年份:2023
- 资助金额:
$ 40.89万 - 项目类别:
Continuing Grant
CAREER: Modeling Genetic Variation Using Deep Generative Neural Networks
职业:使用深度生成神经网络对遗传变异进行建模
- 批准号:
2145577 - 财政年份:2022
- 资助金额:
$ 40.89万 - 项目类别:
Standard Grant
CAREER: Exploiting Low-Dimensional Structures in Data Science: Manifold Learning, Partial Differential Equation Identification, and Neural Networks
职业:在数据科学中利用低维结构:流形学习、偏微分方程识别和神经网络
- 批准号:
2145167 - 财政年份:2022
- 资助金额:
$ 40.89万 - 项目类别:
Continuing Grant
CAREER: Provable Patching of Deep Neural Networks
职业:可证明的深度神经网络修补
- 批准号:
2048123 - 财政年份:2021
- 资助金额:
$ 40.89万 - 项目类别:
Continuing Grant