Phenomenology of Deep Learning

最近机器学习（ML）在工业和科学领域的突破依赖于多层神经网络架构，ML的分支称为深度学习（DL）。在过去的十年中，由于数据集和神经网络的规模令人印象深刻的增长，这些进展在很大程度上是可能的。相反，对深度学习基础的理解并没有遵循这一成功的趋势。事实上，在深度学习的实际成功和我们对深度学习为什么如此有效的理解之间存在着巨大的差距。人们普遍认为，要扩大机器学习的应用范围，获得可靠的人工智能系统，必须对深度学习有一个基本的了解，基于物理的框架可以为深度学习中迫切的问题提供一个独特的视角，有助于填补实践发展与理论基础之间的差距。我们强调，虽然ML应用广泛用于物理学，但思想的流动是相反的，即，使用理论物理学的概念和技术来理解现代深度学习，才刚刚开始探索。在本MSCA中，我们将利用和利用深度神经网络和物理学中的有效场论之间的惊人相似之处。“深度学习现象学”（Phenomenology of Deep Learning，PHENO-DL）的目标是为深度学习的有效理论的发展做出贡献。我们将利用物理模型构建工具来解决基础问题（例如，深度神经网络如何泛化？）以及深度学习中的显著现象（即双下降和对抗性示例）。为此，我们将采用一种受理论物理学中已建立的方法启发的设置，这些方法最近已应用于神经网络。特别是，为了解释神经网络的表现力和能力，我们将使用物理学中的重正化组和深度神经网络中不同层的“特征层次”之间的相互作用。

Recent Machine Learning (ML) breakthroughs in industry and the sciences rely on neural network architectures with multiple layers, abranch of ML called Deep Learning (DL). Over the last decade, much of this progress was possible thanks to the impressive growth ofthe size of datasets and neural networks. In contrast, the understanding of the foundations of DL has not followed this successfultrend. In fact, there is an enormous gap between the practical success of DL and our understanding of why DL works so well. It iswidely acknowledged that to expand the scope of ML applications and obtain reliable artificial intelligence systems, we must achievea fundamental understanding of DL.A physics-based framework can provide a unique perspective to pressing questions in DL and contribute to filling the gap betweenpractical developments and theoretical foundations. We stress that, while ML applications are broadly used in physics, the flow ofideas in the opposite direction, i.e., the use of concepts and techniques from theoretical physics to understand modern deep learning,has only started to be explored.In this MSCA, we will exploit and capitalize on the striking similarities between deep neural networks and effective field theory inphysics. The goal of "Phenomenology of Deep Learning" (PHENO-DL) is to contribute to the development of an effective theory ofdeep learning. We will make use of physics model-building tools to attack foundational questions (e.g., how do deep neural networksgeneralize?) and remarkable phenomena in deep learning (namely, double descent and adversarial examples). To this end, we willadopt a setup inspired by established methods from theoretical physics, which have been recently applied to neural networks. Inparticular, to explain the neural network's expressivity and capacity, we will use the interplay between the renormalisation group inphysics and the 'hierarchy of features' in the different layers in a deep neural network.