Collaborative Research: RI: Small: Theoretical Foundations: The Advantage of Deep Learning over Traditional Shallow Learning Methods

机器学习一直是当前许多智能决策系统背后的主要驱动力。最近，随着越来越多地依赖深度学习方法，它显示出一种范式转变，深度学习方法在图像处理、语音识别、语言翻译和游戏等各种应用中取得了前所未有的性能。除了经验上的成功，可证明的保证和对成功背后原理的洞察也成为人们追求的目标。然而，缺乏足够的理解仍然限制了我们充分利用深度学习潜力的能力。该项目旨在通过了解深度学习相对于传统学习方法的优势，为支持实践趋势奠定基础，这对于揭示实践成功背后的关键因素至关重要。该项目将提供框架，以证明深度学习相对于传统学习方法的性能保证和优势，并使开发更高效、更容易使用的新深度学习方法成为可能。该项目将开发一个全面和系统的方法来理解深度学习优于传统学习方法的经验表现，并使用获得的见解来设计新的学习方法。它将在数据的标记函数和输入结构上开发新的属性理论模型，从而导致实际的成功，并且还提供了证明性能保证和优于浅学习的框架。它还将设计新的学习方法，明确地利用这些属性，从而提高效率和可访问性。尽管最近有重大的研究活动，但这个方向在很大程度上仍未被探索。通过机器学习、统计学和优化工具的跨学科组合，计划的理论和算法解决方案成为可能。提出的方案是基于研究人员之前的工作，包括理论结果和实证验证。如果成功，该研究将为现代智能系统的进一步发展奠定基础，从而具有变革性。它还将有助于解决当前理论没有充分解决的实践中的新理论问题，并将对机器学习和优化产生持久的影响。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning has been a primary driving force behind many current intelligent decision-making systems. Recently it shows a paradigm shift with increasing reliance on deep learning approaches, which have achieved unprecedented performance in various applications such as image processing, speech recognition, language translation, and game playing. Besides the empirical success, provable guarantees and insights into the principles behind the success have also become sought-after goals. However, the lack of adequate understanding is still limiting our capacity to fully exploit the potential of deep learning. This project aims to lay the foundations for supporting the practical trends, by understanding the advantages of deep learning over traditional learning methods, which is crucial for revealing key factors behind the practical success. The project will provide frameworks for proving performance guarantees and advantages of deep learning over traditional learning methods and enable the development of new deep learning methods that are more efficient and accessible. This project will develop a thorough and systematic approach for understanding the superior empirical performance of deep learning over traditional learning methods and use the obtained insights to design new learning methods. It will develop new theoretical models of properties on the labeling function of the data and the structure of the input leading to the practical success, and also provides frameworks for proving performance guarantees and advantages over shallow learning. It will also design new learning methods that explicitly exploit those properties and thus can be more efficient and accessible. This direction is still largely unexplored, despite significant recent research activities. The planned theoretical and algorithmic solutions are possible through an interdisciplinary mix of tools from machine learning, statistics, and optimization. The proposed program is grounded in the investigators' prior work that includes both theoretical results and empirical validation. If successful, the proposed research can be transformational for modern intelligent systems by laying the foundations for further development. It will also help to solve new theoretical problems from practice that are not adequately addressed by current theory and will have lasting impacts on machine learning and optimization.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.