Collaborative Research: New Perspectives on Deep Learning: Bridging Approximation, Statistical, and Algorithmic Theories
合作研究:深度学习的新视角:桥接近似、统计和算法理论
基本信息
- 批准号:2134140
- 负责人:
- 金额:$ 22.5万
- 依托单位:
- 依托单位国家:美国
- 项目类别:Standard Grant
- 财政年份:2021
- 资助国家:美国
- 起止时间:2021-11-01 至 2024-10-31
- 项目状态:已结题
- 来源:
- 关键词:
项目摘要
Deep Learning (DL) has led to a renaissance in neural network methods in data-driven science and engineering. The development of DL systems and applications, including computer vision and natural language understanding, has been led primarily by experiments and engineering practice. Mathematical analysis has only begun to provide insights into these complex machine learning systems. The lack of basic understanding has contributed to serious challenges and shortcomings ranging from the fragility and susceptibility to corrupted data to their uninterpretable behaviors. These problems can be traced to fundamental gaps in the mathematical understanding of DL. This project tackles this challenge by bringing approximation, statistical, and algorithmic theories together to develop new mathematical foundations for DL. The goals of the project are to mathematically characterize the strengths and limitations of DL models, and to understand the properties of DL models trained using examples of desired behavior (training data) as well as the tradeoffs between the performance of DL systems and the training dataset size. While DL is already in widespread use, the continued success of DL requires far more complete mathematical understandings and principled approaches to guide its use and reliable application. The project will provide practitioners with clearer guidance on the strengths, limitations, and best approaches to using DL. Broader impacts of the project also include education and mentoring, including the training of graduate students in mathematical fields such as approximation theory, signal processing, statistics, and machine learning and, most importantly, how these fields collectively inform the theory and practice of DL.DL seeks to learn an unknown function from data using compositions (layers) of linear combinations of simple functions (neurons). The shortcomings of DL can be traced to fundamental gaps in its mathematical theory including the following issues. The function spaces that capture the salient properties of DL applications are poorly understood. The characteristics of functions learned through neural network training are mysterious. The ability of DL models to discriminate between data distributions has not yet been quantified satisfactorily. Understanding of the tradeoffs between accuracy and training set size is lacking. This project tackles these challenges by bringing approximation, statistical, and algorithmic theories together to develop new theoretical foundations for DL. This project builds innovative bridges between approximation theory, nonparametric statistics, learning theory and algorithms to develop new mathematical foundations for DL. This includes the development of new model classes of functions that are naturally suited to characterize the properties, strengths, and limitations of deep neural network architectures and applications; novel approaches to understand the roles of regularization and sparsity in DL; fundamental frameworks to quantify the discrimination power of DL and generalized adversarial networks; and innovative theory to make DL algorithms more data efficient through the use of side-information, partial differential equations, and richer forms of data than the conventional function evaluations.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
深度学习(DL)导致了数据驱动的科学和工程中神经网络方法的复兴。深度学习系统和应用的开发,包括计算机视觉和自然语言理解,主要由实验和工程实践主导。数学分析才刚刚开始为这些复杂的机器学习系统提供见解。缺乏基本的理解导致了严重的挑战和缺陷,从脆弱性和易受损坏的数据到无法解释的行为。 这些问题可以追溯到对深度学习的数学理解的根本差距。该项目通过将近似,统计和算法理论结合在一起来解决这一挑战,为DL开发新的数学基础。该项目的目标是在数学上表征DL模型的优势和局限性,并了解使用所需行为(训练数据)的示例训练的DL模型的属性,以及DL系统的性能和训练数据集大小之间的权衡。虽然深度学习已经被广泛使用,但深度学习的持续成功需要更完整的数学理解和原则性方法来指导其使用和可靠的应用。该项目将为从业者提供关于使用DL的优势,限制和最佳方法的更清晰的指导。该项目的更广泛的影响还包括教育和指导,包括在数学领域(如近似理论,信号处理,统计和机器学习)的研究生培训,最重要的是,这些领域如何共同告知DL的理论和实践。DL试图使用简单函数(神经元)的线性组合的组合(层)从数据中学习未知函数。DL的缺点可以追溯到其数学理论中的基本空白,包括以下问题。捕获DL应用程序的显着属性的函数空间知之甚少。通过神经网络训练学习的函数的特征是神秘的。DL模型区分数据分布的能力尚未得到令人满意的量化。缺乏对准确性和训练集大小之间的权衡的理解。该项目通过将近似,统计和算法理论结合在一起来解决这些挑战,为DL开发新的理论基础。该项目在近似理论,非参数统计,学习理论和算法之间建立了创新的桥梁,为DL开发新的数学基础。这包括开发新的函数模型类,这些函数自然适合于表征深度神经网络架构和应用的属性、优势和局限性;理解正则化和稀疏性在深度神经网络中的作用的新方法;量化深度神经网络和广义对抗网络的区分能力的基本框架;和创新理论,通过使用边信息,偏微分方程,该奖项反映了NSF的法定使命,并被认为是值得支持的,使用基金会的知识价值和更广泛的影响审查标准进行评估。
项目成果
期刊论文数量(5)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Active Learning with Neural Networks: Insights from Nonparametric Statistics
- DOI:10.48550/arxiv.2210.08367
- 发表时间:2022-10
- 期刊:
- 影响因子:0
- 作者:Yinglun Zhu;R. Nowak
- 通讯作者:Yinglun Zhu;R. Nowak
A Continuous Transform for Localized Ridgelets
局部脊波的连续变换
- DOI:
- 发表时间:2023
- 期刊:
- 影响因子:0
- 作者:Shenouda;Joseph;Parhi, Rahul;Nowak, Robert
- 通讯作者:Nowak, Robert
Pure Exploration in Kernel and Neural Bandits
内核和神经强盗的纯粹探索
- DOI:
- 发表时间:2021
- 期刊:
- 影响因子:0
- 作者:Zhu, Yinglun;Zhou, Dongruo;Jiang, Ruoxi;Gu, Quanquan;Willett, Rebecca;Nowak, Robert
- 通讯作者:Nowak, Robert
What Kinds of Functions Do Deep Neural Networks Learn? Insights from Variational Spline Theory
深度神经网络学习哪些类型的函数?
- DOI:10.1137/21m1418642
- 发表时间:2022
- 期刊:
- 影响因子:3.6
- 作者:Parhi, Rahul;Nowak, Robert D.
- 通讯作者:Nowak, Robert D.
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Robert Nowak其他文献
Lock-free de Bruijn graph
无锁 de Bruijn 图
- DOI:
10.48550/arxiv.2401.02756 - 发表时间:
2024 - 期刊:
- 影响因子:0
- 作者:
Daniel G'orniak;Robert Nowak - 通讯作者:
Robert Nowak
NIH consensus conference. Adjuvant therapy for patients with colon and rectal cancer.
NIH 共识会议。
- DOI:
- 发表时间:
1990 - 期刊:
- 影响因子:0
- 作者:
Jeongyeol Kwon;Liu Yang;Robert Nowak;Josiah P. Hanna - 通讯作者:
Josiah P. Hanna
On Regret with Multiple Best Arms
多臂后悔
- DOI:
- 发表时间:
2020 - 期刊:
- 影响因子:0
- 作者:
Yinglun Zhu;Robert Nowak - 通讯作者:
Robert Nowak
Future Prediction Can be a Strong Evidence of Good History Representation in Partially Observable Environments
未来预测可以成为部分可观测环境中良好历史表征的有力证据
- DOI:
- 发表时间:
2024 - 期刊:
- 影响因子:0
- 作者:
Jeongyeol Kwon;Liu Yang;Robert Nowak;Josiah P. Hanna - 通讯作者:
Josiah P. Hanna
Looped Transformers are Better at Learning Learning Algorithms
循环变压器更擅长学习学习算法
- DOI:
- 发表时间:
2023 - 期刊:
- 影响因子:0
- 作者:
Liu Yang;Kangwook Lee;Robert Nowak;Dimitris Papailiopoulos - 通讯作者:
Dimitris Papailiopoulos
Robert Nowak的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Robert Nowak', 18)}}的其他基金
CIF: Small: Bridging the Inequality Gap
CIF:小:缩小不平等差距
- 批准号:
1907786 - 财政年份:2019
- 资助金额:
$ 22.5万 - 项目类别:
Standard Grant
Collaborative Research: Physics-Based Machine Learning for Sub-Seasonal Climate Forecasting
合作研究:基于物理的机器学习用于次季节气候预测
- 批准号:
1934612 - 财政年份:2019
- 资助金额:
$ 22.5万 - 项目类别:
Continuing Grant
EAGER: Developing a Theory for Function Optimization on Graphs Using Local Information
EAGER:开发使用局部信息的图函数优化理论
- 批准号:
1841190 - 财政年份:2018
- 资助金额:
$ 22.5万 - 项目类别:
Standard Grant
BIGDATA: F: DKA: CSD: Human and Machine Co-Processing
BIGDATA:F:DKA:CSD:人机协同处理
- 批准号:
1447449 - 财政年份:2014
- 资助金额:
$ 22.5万 - 项目类别:
Standard Grant
CIF: Small: Sparsity and Scarcity in High-Dimensional Point Processes
CIF:小:高维点过程中的稀疏性和稀缺性
- 批准号:
1418976 - 财政年份:2013
- 资助金额:
$ 22.5万 - 项目类别:
Standard Grant
CIF: Small: Adaptive Information: Sequential Sensing and Active Learning Theory, Methods and Applications
CIF:小型:自适应信息:顺序感知和主动学习理论、方法和应用
- 批准号:
1218189 - 财政年份:2012
- 资助金额:
$ 22.5万 - 项目类别:
Standard Grant
CIF: Small: Decoding Error-Correcting Codes using Large-Scale Decomposition Methods
CIF:小型:使用大规模分解方法解码纠错码
- 批准号:
1217058 - 财政年份:2012
- 资助金额:
$ 22.5万 - 项目类别:
Standard Grant
CIF: Medium: Collaborative Research: Cooperative Routing in Wireless Ad-Hoc Networks with Advanced PHY Layers: Interference Management, Resource Allocation, and Information Mixing
CIF:中:协作研究:具有高级 PHY 层的无线 Ad-Hoc 网络中的协作路由:干扰管理、资源分配和信息混合
- 批准号:
0963834 - 财政年份:2010
- 资助金额:
$ 22.5万 - 项目类别:
Continuing Grant
EAGER: Building Arid-land International Collaborations between US and China: Ecology of Invasive Plants
EAGER:中美之间建立旱地国际合作:入侵植物生态学
- 批准号:
1047575 - 财政年份:2010
- 资助金额:
$ 22.5万 - 项目类别:
Standard Grant
相似国自然基金
Research on Quantum Field Theory without a Lagrangian Description
- 批准号:24ZR1403900
- 批准年份:2024
- 资助金额:0.0 万元
- 项目类别:省市级项目
Cell Research
- 批准号:31224802
- 批准年份:2012
- 资助金额:24.0 万元
- 项目类别:专项基金项目
Cell Research
- 批准号:31024804
- 批准年份:2010
- 资助金额:24.0 万元
- 项目类别:专项基金项目
Cell Research (细胞研究)
- 批准号:30824808
- 批准年份:2008
- 资助金额:24.0 万元
- 项目类别:专项基金项目
Research on the Rapid Growth Mechanism of KDP Crystal
- 批准号:10774081
- 批准年份:2007
- 资助金额:45.0 万元
- 项目类别:面上项目
相似海外基金
Collaborative Research: REU Site: Earth and Planetary Science and Astrophysics REU at the American Museum of Natural History in Collaboration with the City University of New York
合作研究:REU 地点:地球与行星科学和天体物理学 REU 与纽约市立大学合作,位于美国自然历史博物馆
- 批准号:
2348998 - 财政年份:2025
- 资助金额:
$ 22.5万 - 项目类别:
Standard Grant
Collaborative Research: REU Site: Earth and Planetary Science and Astrophysics REU at the American Museum of Natural History in Collaboration with the City University of New York
合作研究:REU 地点:地球与行星科学和天体物理学 REU 与纽约市立大学合作,位于美国自然历史博物馆
- 批准号:
2348999 - 财政年份:2025
- 资助金额:
$ 22.5万 - 项目类别:
Standard Grant
Collaborative Research: Resolving the LGM ventilation age conundrum: New radiocarbon records from high sedimentation rate sites in the deep western Pacific
合作研究:解决LGM通风年龄难题:西太平洋深部高沉降率地点的新放射性碳记录
- 批准号:
2341426 - 财政年份:2024
- 资助金额:
$ 22.5万 - 项目类别:
Continuing Grant
Collaborative Research: Resolving the LGM ventilation age conundrum: New radiocarbon records from high sedimentation rate sites in the deep western Pacific
合作研究:解决LGM通风年龄难题:西太平洋深部高沉降率地点的新放射性碳记录
- 批准号:
2341424 - 财政年份:2024
- 资助金额:
$ 22.5万 - 项目类别:
Continuing Grant
Collaborative Research: New to IUSE: EDU DCL:Diversifying Economics Education through Plug and Play Video Modules with Diverse Role Models, Relevant Research, and Active Learning
协作研究:IUSE 新增功能:EDU DCL:通过具有不同角色模型、相关研究和主动学习的即插即用视频模块实现经济学教育多元化
- 批准号:
2315700 - 财政年份:2024
- 资助金额:
$ 22.5万 - 项目类别:
Standard Grant
Collaborative Research: AF: Small: New Directions in Algorithmic Replicability
合作研究:AF:小:算法可复制性的新方向
- 批准号:
2342244 - 财政年份:2024
- 资助金额:
$ 22.5万 - 项目类别:
Standard Grant
Collaborative Research: On New Directions for the Derivation of Wave Kinetic Equations
合作研究:波动力学方程推导的新方向
- 批准号:
2306378 - 财政年份:2024
- 资助金额:
$ 22.5万 - 项目类别:
Standard Grant
Collaborative Research: New to IUSE: EDU DCL:Diversifying Economics Education through Plug and Play Video Modules with Diverse Role Models, Relevant Research, and Active Learning
协作研究:IUSE 新增功能:EDU DCL:通过具有不同角色模型、相关研究和主动学习的即插即用视频模块实现经济学教育多元化
- 批准号:
2315699 - 财政年份:2024
- 资助金额:
$ 22.5万 - 项目类别:
Standard Grant
Collaborative Research: Understanding New Labor Relations for the 21st Century
合作研究:理解21世纪的新型劳动关系
- 批准号:
2346230 - 财政年份:2024
- 资助金额:
$ 22.5万 - 项目类别:
Standard Grant
Collaborative Research: New Regression Models and Methods for Studying Multiple Categorical Responses
合作研究:研究多重分类响应的新回归模型和方法
- 批准号:
2415067 - 财政年份:2024
- 资助金额:
$ 22.5万 - 项目类别:
Continuing Grant