Phenomenology of Deep Learning

最近机器学习（ML）在工业和科学领域的突破依赖于多层神经网络架构，这是机器学习的一个分支，称为深度学习（DL）。在过去的十年中，由于数据集和神经网络规模的惊人增长，这种进步在很大程度上是可能的。相比之下，对深度学习基础的理解并没有遵循这一成功的趋势。事实上，深度学习的实际成功与我们对深度学习为何如此有效的理解之间存在着巨大的差距。人们普遍认为，要扩大机器学习的应用范围并获得可靠的人工智能系统，我们必须对机器学习有一个基本的了解。基于物理的框架可以为深度学习中的紧迫问题提供独特的视角，并有助于填补实践发展与理论基础之间的差距。我们强调，虽然机器学习应用广泛应用于物理学，但相反方向的思想流，即使用理论物理学的概念和技术来理解现代深度学习，才刚刚开始探索。在这个MSCA中，我们将利用和利用深度神经网络和物理中的有效场论之间惊人的相似性。“深度学习现象学”（PHENO-DL）的目标是促进有效的深度学习理论的发展。我们将利用物理模型构建工具来解决基础问题（例如，深度神经网络如何泛化？）和深度学习中的显著现象（即双重下降和对抗性示例）。为此，我们将采用一种受理论物理中已建立的方法启发的设置，这些方法最近已应用于神经网络。特别是，为了解释神经网络的表现力和容量，我们将使用物理学中的重整化群与深度神经网络中不同层中的“特征层次”之间的相互作用。

Recent Machine Learning (ML) breakthroughs in industry and the sciences rely on neural network architectures with multiple layers, abranch of ML called Deep Learning (DL). Over the last decade, much of this progress was possible thanks to the impressive growth ofthe size of datasets and neural networks. In contrast, the understanding of the foundations of DL has not followed this successfultrend. In fact, there is an enormous gap between the practical success of DL and our understanding of why DL works so well. It iswidely acknowledged that to expand the scope of ML applications and obtain reliable artificial intelligence systems, we must achievea fundamental understanding of DL.A physics-based framework can provide a unique perspective to pressing questions in DL and contribute to filling the gap betweenpractical developments and theoretical foundations. We stress that, while ML applications are broadly used in physics, the flow ofideas in the opposite direction, i.e., the use of concepts and techniques from theoretical physics to understand modern deep learning,has only started to be explored.In this MSCA, we will exploit and capitalize on the striking similarities between deep neural networks and effective field theory inphysics. The goal of "Phenomenology of Deep Learning" (PHENO-DL) is to contribute to the development of an effective theory ofdeep learning. We will make use of physics model-building tools to attack foundational questions (e.g., how do deep neural networksgeneralize?) and remarkable phenomena in deep learning (namely, double descent and adversarial examples). To this end, we willadopt a setup inspired by established methods from theoretical physics, which have been recently applied to neural networks. Inparticular, to explain the neural network's expressivity and capacity, we will use the interplay between the renormalisation group inphysics and the 'hierarchy of features' in the different layers in a deep neural network.