CAREER: Towards a Theory of Deep Learning

在过去的十年里，深度学习已经从征服研究基准发展到每天与人类互动的系统，包括机器翻译、医疗保健、语音识别、半自动驾驶汽车和自动化。然而，它在经验上的成功与对它为什么/何时起作用的理论理解之间存在着巨大的差距。该项目旨在通过对深度学习的基本理解和设计算法来提高可靠性和数据效率，从而缩小这一差距。更广泛地说，这个项目的社会影响包括i)与机器学习相关的算法的理论理解和设计，ii)为中学教师开发新的系列研讨会和讲习班的教育计划，以及iii)改善学术界的残疾人住宿。这个项目分为三个不同的重点。第一个重点是理解算法和架构的算法正则化效应。利用这些见解，团队将设计更好的损失函数和架构，以提高准确性。第二个重点是从理论上比较用随机梯度下降训练的网络与它们的结构诱导核方法的准确性。这种比较可能在理论上证明神经网络可以进行特征学习，这解释了深度学习在经验上的成功，而核方法不能。最后，该项目将研究表征学习，并从理论上分析深度网络如何在不同领域之间转移其表征。这种转移将减少深度学习对标记数据的需求，可能使其应用于数据匮乏的领域。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Over the past decade, deep learning has evolved from conquering research benchmarks to systems that interact with humans on a daily basis, including in machine translation, healthcare, speech recognition, semi-autonomous vehicles, and automation. However, a large gap exists between its empirical successes and a theoretical understanding of why / when it works. This project aims to close this gap through foundational understanding of deep learning and designing algorithms to improve reliability and data efficiency. More broadly, the societal impact of this project include i) theoretical understanding and design of algorithms relevant to machine learning, ii) education plans that develop a new seminar series and workshops for secondary school teachers, and iii) improving disability accommodation in academia.This project is divided into three different thrusts. The first thrust is to understand the algorithmic regularization effect of algorithms and architectures. Using these insights, the team will design better loss functions and architectures to improve accuracy. The second thrust is to theoretically compare the accuracy of networks trained with stochastic gradient descent against their architecture-induced kernel methods. This comparison may theoretically demonstrate that neural networks can do feature learning, which explains the empirical success of deep learning, and that kernel methods cannot. Finally, the project will study representation learning, and theoretically analyze how deep networks can transfer their representations between different domains. Such a transfer will allow a reduction of the labeled data requirements for deep learning, potentially allowing its application to data-starved domains.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.