RI: AF: Small: Deep Learning Theory

深度学习最近成为机器学习和人工智能领域的一大进步。这种从数据中学习的技术为图像分类和语音识别提供了改变领域的性能改进，它在各种领域（包括自然语言处理、机器人技术、音频处理和计算化学）都显示出令人印象深刻的性能，并且它已成为人工智能系统的核心成分。但是，尽管取得了这些成功，我们对这些方法的理解是不完整的。本研究项目的总体目标是解决这一重大挑战：开发分析技术，使我们能够理解深度学习方法何时以及为什么会成功，并设计具有明确性能保证的有效方法。成功的研究成果在使用这些方法的大量和不断增长的应用领域具有重大的实际影响潜力。该项目旨在了解深度学习方法的性能，特别是阐明哪些方面对其成功至关重要，从而开发有原则的设计技术和性能保证。目标是：描述当前神经网络架构的关键特征对性能的影响：规模、深度、非线性和正则化；开发有助于我们理解深层架构的近似和估计特性的分析技术；识别深度网络易学习问题和难学习问题的边界；并开发具有显式性能保证的方法来优化深度神经网络。成功的研究成果可能会增加我们对深度学习方法的理解，为这些方法提供性能保证，并促进新型深度学习方法的原则性设计。

Deep learning has recently emerged as a major advance in machine learning and AI. This technology for learning from data has provided field-changing performance improvements in image classification and speech recognition, it has displayed impressive performance across a large variety of areas (including natural language processing, robotics, audio processing, and computational chemistry), and it has become a central ingredient in AI systems. But despite these successes, our understanding of these methods is incomplete. The broad goal of this research project is to address this grand challenge: to develop analysis techniques that enable us to understand when and why deep learning methods will be successful, and to design effective methods with explicit performance guarantees. Successful research outcomes have a significant potential for practical impact in the large and growing set of application areas where these methods are used.The project aims to understand the performance of deep learning methods - in particular to elucidate what aspects are essential for their success - and hence to develop principled design techniques and performance guarantees. The objectives are: to characterize the performance impacts of the critical features of current neural network architectures: scale, depth, nonlinearities, and regularization; to develop analysis techniques that facilitate our understanding of the approximation and estimation properties of deep architectures; to identify the boundary between easy and hard learning problems for deep networks; and to develop methods with explicit performance guarantees for optimization in deep neural networks. Successful research outcomes are likely to increase our understanding of deep learning methods, to provide performance guarantees for these methods, and to facilitate the principled design of novel deep learning methods.