Geometrization Approaches toward Understanding Deep Learning

该项目将研究深度学习的理论基础，深度学习是一种广泛成功的方法，用于各种数据广泛的应用。对深度学习的全面理解对于开发深度学习模型的原则性设计和训练至关重要，最终减少计算负担和人力成本。该研究将涵盖三个基础和互补的方向：理解训练有素的深度神经网络，研究深度学习训练动态，以及探索深度学习模型内部层的数据处理。通过弥合现代神经网络的复杂训练范式与现有理论之间的差距，该项目将揭开这些黑箱模型的神秘面纱，使其在广泛的科学和工程应用中更具可解释性和效率。该项目将提供多个跨学科的机会，以促进下一代统计学家和数据科学家的专业发展。研究活动将集中在三个研究项目上，从统计和数学的角度全面了解深度学习。第一个项目将在深度学习训练的最后阶段分析对称几何，开发新的优化技术和统计见解。第二个项目将详细了解现代深度学习训练的动态，将统计和推理思想融入深度学习动态的活跃领域。第三个项目将研究深度学习如何使用随机矩阵理论、非凸优化和学习理论的技术，根据神经网络所有层的类成员关系来分离数据。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The project will study the theoretical underpinnings of deep learning, a widely successful approach to various data-extensive applications. A comprehensive understanding of deep learning is crucial for the development of principled design and training of deep learning models, ultimately reducing computational burden and human costs. The research will span three foundational and complementary directions: understanding well-trained deep neural networks, examining deep learning training dynamics, and exploring data processing in interior layers of deep learning models. By bridging the gap between the complex training paradigms of modern neural networks and existing theories, the project will demystify these black-box models, making them more interpretable and efficient for a wide range of scientific and engineering applications. The project will offer multiple interdisciplinary opportunities for boosting the professional development of the next generation of statisticians and data scientists. The research activities will focus on three research projects to develop a comprehensive understanding of deep learning from a statistical and mathematical perspective. The first project will analyze symmetric geometries in the final stages of deep learning training, developing novel optimization techniques and statistical insights. The second project will provide a detailed understanding of the dynamics of modern deep learning training, integrating statistical and inferential ideas into the active field of deep learning dynamics. The third project will investigate how deep learning separates data according to class membership across all layers of the neural network, using techniques from random matrix theory, non-convex optimization, and learning theory. The successful completion of these research projects will result in a geometrization of deep learning methodologies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.