CAREER: Towards a Theory of Deep Learning

职业：走向深度学习理论

• 批准号: 2144994
• 负责人: Jason Lee
• 金额: $ 58.3万
• 依托单位: Princeton University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-06-01 至 2027-05-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2144994&HistoricalAwards=false
• 关键词: CAREER; Towards; Theory; Deep; Learning

Over the past decade, deep learning has evolved from conquering research benchmarks to systems that interact with humans on a daily basis, including in machine translation, healthcare, speech recognition, semi-autonomous vehicles, and automation. However, a large gap exists between its empirical successes and a theoretical understanding of why / when it works. This project aims to close this gap through foundational understanding of deep learning and designing algorithms to improve reliability and data efficiency. More broadly, the societal impact of this project include i) theoretical understanding and design of algorithms relevant to machine learning, ii) education plans that develop a new seminar series and workshops for secondary school teachers, and iii) improving disability accommodation in academia.This project is divided into three different thrusts. The first thrust is to understand the algorithmic regularization effect of algorithms and architectures. Using these insights, the team will design better loss functions and architectures to improve accuracy. The second thrust is to theoretically compare the accuracy of networks trained with stochastic gradient descent against their architecture-induced kernel methods. This comparison may theoretically demonstrate that neural networks can do feature learning, which explains the empirical success of deep learning, and that kernel methods cannot. Finally, the project will study representation learning, and theoretically analyze how deep networks can transfer their representations between different domains. Such a transfer will allow a reduction of the labeled data requirements for deep learning, potentially allowing its application to data-starved domains.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在过去的十年中，深度学习从征服研究基准演变为每天与人类互动的系统，包括在机器翻译，医疗保健，语音识别，半自动车辆和自动化中。但是，其经验成功与对为什么 /何时起作用的理论理解之间存在很大的差距。该项目旨在通过对深度学习和设计算法的基本理解来缩小这一差距，以提高可靠性和数据效率。更广泛地说，该项目的社会影响包括i）与机器学习相关的算法的理论理解和设计，ii）为中学教师开发新的研讨会系列和讲习班的教育计划，以及iii）改善学术界的残疾人住宿。这项项目分为三种不同的推力。第一个推力是了解算法和体系结构的算法正则化效果。使用这些见解，团队将设计更好的损失功能和架构以提高准确性。第二个推力是从理论上比较了用随机梯度下降训练的网络与构建诱导的内核方法的准确性。从理论上讲，这种比较可以证明神经网络确实具有学习能力，从而解释了深度学习的经验成功，而内核方法不能。最后，该项目将研究表示形式学习，并理论上分析了深层网络如何在不同领域之间传输其表示形式。这样的转移将允许减少深度学习的标记数据要求，从而有可能将其应用于数据饥饿的领域。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子和更广泛影响的评估审查标准来评估的值得支持的。