权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

CAREER: Optimization Landscape for Non-convex Functions - Towards Provable Algorithms for Neural Networks

职业：非凸函数的优化景观 - 走向可证明的神经网络算法

基本信息

批准号：
1845171
负责人：
Rong Ge
金额：
$ 40万
依托单位：
Duke University
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2019
资助国家：
美国
起止时间：
2019-07-01 至 2024-06-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1845171&HistoricalAwards=false
关键词：
CAREER Optimization Landscape Non convex

项目摘要

Deep learning, a machine learning method that is based on artificial neural networks, has greatly improved the performance of learning algorithms for many tasks that are related to understanding complicated data such as natural images, videos and language. Products based on deep learning have already made real-life impact in face recognition, machine translation, and shown promise for more applications such as self-driving cars. However, despite the practical success of deep learning, theoretical understanding for why these algorithms work has been scarce. One of the main difficulties in understanding deep learning algorithms is that these algorithms need to solve very complicated optimization problems that try to find out what are the best ways for the neurons to be connected. In the most general form, these optimization problems are known to be intractable. This research project will identify properties of the real-world problems that make these problems special and tractable, and provide new optimization algorithms with theoretical guarantees that are applicable to deep learning. The materials developed in the project will be disseminated through conferences and workshops that try to connect different research communities, and used to create new machine learning courses. The algorithms designed in the project will also be implemented in standard deep learning frameworks and made publicly available.The specific approach of this project revolves around the new concept of optimization landscape. For an optimization problem, its optimization landscape includes clear understanding of the location and values of its local and global optimal solutions. The research goals are divided into three categories. First, the research project will focus on a class of locally optimizable functions for which local minima are all globally optimal. The research project will develop simple and efficient algorithms for optimizing such functions, as well as a new framework to prove several problems of practical interest are locally optimizable. Second, the project will develop stronger optimization algorithms that can work even when the optimization landscape is not as ideal. Finally, the research will focus on optimization problems that arise in deep learning and show how the techniques developed in the previous two parts can be applied. These projects will bring more theoretical insights into the heuristics for training neural networks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度学习是一种基于人工神经网络的机器学习方法，它极大地提高了与理解自然图像、视频和语言等复杂数据相关的许多任务的学习算法的性能。基于深度学习的产品已经在人脸识别、机器翻译等领域产生了现实影响，并有望在自动驾驶汽车等更多应用中发挥作用。然而，尽管深度学习在实践中取得了成功，但对这些算法为什么有效的理论理解却很少。理解深度学习算法的主要困难之一是，这些算法需要解决非常复杂的优化问题，试图找出神经元连接的最佳方式。在最一般的形式，这些优化问题是已知的是棘手的。该研究项目将确定现实世界问题的属性，使这些问题变得特殊和易于处理，并提供新的优化算法，并提供适用于深度学习的理论保证。该项目开发的材料将通过会议和研讨会传播，这些会议和研讨会试图连接不同的研究社区，并用于创建新的机器学习课程。该项目中设计的算法也将在标准深度学习框架中实现并公开提供。该项目的具体方法围绕优化景观的新概念。对于优化问题，其优化景观包括对其局部和全局最优解的位置和值的清晰理解。研究目标分为三类。首先，本研究计画将集中于一类局部最佳化函数，其局部最小值皆为全局最佳。该研究项目将开发简单有效的算法来优化这些函数，以及一个新的框架来证明几个实际问题是局部优化的。其次，该项目将开发更强大的优化算法，即使在优化环境不理想的情况下也可以工作。最后，本研究将重点关注深度学习中出现的优化问题，并展示如何应用前两部分中开发的技术。这些项目将为训练神经网络的算法带来更多的理论见解。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。