Phenomenology of Deep Learning

最近机器学习(ML)在工业和科学方面的突破依赖于具有多层的神经网络结构，ML的分支称为深度学习(DL)。在过去的十年里，由于数据集和神经网络的规模令人印象深刻的增长，这一进步在很大程度上成为可能。相比之下，对数字图书馆基础的理解并没有跟随这一成功的趋势。事实上，在数字图书馆的实际成功与我们对数字图书馆为何如此成功的理解之间存在着巨大的差距。人们普遍认为，要扩大ML的应用范围并获得可靠的人工智能系统，我们必须对DL有一个基本的理解。一个基于物理的框架可以为DL中迫切的问题提供一个独特的视角，并有助于填补实践发展和理论基础之间的差距。我们强调，虽然ML应用程序在物理学中得到了广泛的应用，但相反方向的思想流动，即使用理论物理学的概念和技术来理解现代深度学习，才刚刚开始探索。在本MSCA中，我们将利用深层神经网络和物理学中有效场论之间的惊人相似之处。“深度学习现象学”的目标是为发展一种有效的深度学习理论做出贡献。我们将利用物理建模工具来解决基本问题(例如，深度神经网络如何泛化？)深度学习中的显著现象(即双重下降和对抗性例子)。为此，我们将采用一种受到理论物理既定方法启发的设置，这些方法最近已应用于神经网络。特别是，为了解释神经网络的表达能力和能力，我们将使用物理中的重正化组和深层神经网络中不同层的“特征层次”之间的相互作用。

Recent Machine Learning (ML) breakthroughs in industry and the sciences rely on neural network architectures with multiple layers, abranch of ML called Deep Learning (DL). Over the last decade, much of this progress was possible thanks to the impressive growth ofthe size of datasets and neural networks. In contrast, the understanding of the foundations of DL has not followed this successfultrend. In fact, there is an enormous gap between the practical success of DL and our understanding of why DL works so well. It iswidely acknowledged that to expand the scope of ML applications and obtain reliable artificial intelligence systems, we must achievea fundamental understanding of DL.A physics-based framework can provide a unique perspective to pressing questions in DL and contribute to filling the gap betweenpractical developments and theoretical foundations. We stress that, while ML applications are broadly used in physics, the flow ofideas in the opposite direction, i.e., the use of concepts and techniques from theoretical physics to understand modern deep learning,has only started to be explored.In this MSCA, we will exploit and capitalize on the striking similarities between deep neural networks and effective field theory inphysics. The goal of "Phenomenology of Deep Learning" (PHENO-DL) is to contribute to the development of an effective theory ofdeep learning. We will make use of physics model-building tools to attack foundational questions (e.g., how do deep neural networksgeneralize?) and remarkable phenomena in deep learning (namely, double descent and adversarial examples). To this end, we willadopt a setup inspired by established methods from theoretical physics, which have been recently applied to neural networks. Inparticular, to explain the neural network's expressivity and capacity, we will use the interplay between the renormalisation group inphysics and the 'hierarchy of features' in the different layers in a deep neural network.