权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

CAREER: Theoretical foundations of neural networks - representation, optimization, and generalization

职业：神经网络的理论基础——表示、优化和泛化

基本信息

批准号：
1750051
负责人：
Matus Telgarsky
金额：
$ 50万
依托单位：
University of Illinois at Urbana-Champaign
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2018
资助国家：
美国
起止时间：
2018-03-15 至 2024-02-29
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1750051&HistoricalAwards=false
关键词：
CAREER Theoretical foundations neural networks

项目摘要

Neural networks form the backbone of machine learning's recent advances and sudden ubiquity. Despite this extensive empirical progress, however, a satisfactory understanding of their behavior is still missing. As neural networks enter more and more into human-facing services (self-driving cars, medical diagnostics, etc.), this status quo and in particular its safety ramifications becomes worrisome. This project aims for a theoretical understanding of the foundations of neural networks, divided into three pieces: (a) the representation question regarding which phenomena can be succinctly approximated by neural networks; (b) the optimization question of how to efficiently fit neural networks to data; and (c) the generalization question on why neural networks can fit not only the data they have seen but also the data they have not seen. Developing this understanding will form the core of this project's three broader impacts: (1) the research component will aim to improve safety and reliability of user-facing deployments of neural networks; (2) as an educational component, the research will be simplified and incorporated into freely available course notes; (3) the award supports two outreach efforts co-founded by the PI: UIUC-ML, a university-wide ML seminar; and the midwest ML symposium, a yearly midwest ML gathering.In more detail, the technical focus of this project, divided into the three learning theoretic topics above, is as follows. The core representation question is: what makes neural network representation special? In more detail, the proposed representation questions are firstly to characterize the power gained by adding a single layer to a network, and secondly to characterize the representation properties of recurrent neural networks, namely neural networks which evolve their state along with a time series they consume. Next comes the topic of optimization, where the key mystery is how neural networks manage to perfectly fit their data with simple iterative descent schemes, despite the apparent nonconvexity of the problem. The plan here is to establish an even stronger property: these iterative schemes manage to output networks which not only fit their data, but do so confidently, in the classical sense of margin theory. Finally, the proposal closes with the topic of generalization. The first goal is to develop refined generalization bounds to the point that they can be algorithmically enforced via effective regularization schemes, and secondarily to apply these techniques to the fitting of neural networks to probability distributions, specifically the problem of training Generative Adversarial Networks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

神经网络是机器学习最近取得的进步和突然普及的支柱。然而，尽管取得了广泛的经验性进展，对它们的行为仍然缺乏令人满意的理解。随着神经网络越来越多地进入面向人类的服务（自动驾驶汽车、医疗诊断等），这一现状，特别是其安全后果令人担忧。本课题的目的是从理论上理解神经网络的基础，分为三个部分：（a）关于哪些现象可以用神经网络简洁地近似的表示问题;（B）如何有效地使神经网络与数据相适应的优化问题;以及（c）关于为什么神经网络不仅可以拟合它们已经看到的数据，还可以拟合它们还没有看到的数据的泛化问题。发展这种理解将构成该项目三个更广泛影响的核心：（1）研究部分旨在提高面向用户的神经网络部署的安全性和可靠性;（2）作为教育部分，研究将被简化并纳入免费课程笔记;（3）该奖项支持PI共同创立的两项外展工作：IUC-ML，全校范围的ML研讨会;以及中西部ML研讨会，一年一度的中西部ML聚会。更详细地说，该项目的技术重点分为上述三个学习理论主题，如下所示。核心的表示问题是：是什么让神经网络表示特别？更详细地说，所提出的表示问题首先是表征通过向网络添加单层而获得的功率，其次是表征循环神经网络的表示属性，即沿着它们消耗的时间序列发展其状态的神经网络。接下来是优化的主题，其中关键的奥秘是神经网络如何设法用简单的迭代下降方案完美地拟合其数据，尽管问题明显是非凸的。这里的计划是建立一个更强的属性：这些迭代方案设法输出网络，这些网络不仅适合它们的数据，而且在经典意义上的边际理论中自信地这样做。最后，该提案以推广的主题结束。第一个目标是开发精细的泛化边界，使其可以通过有效的正则化方案在算法上强制执行，其次是将这些技术应用于神经网络对概率分布的拟合，该奖项反映了NSF的法定使命，并被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准。