Phenomenology of Deep Learning

点击翻译按钮获取中文摘要

Recent Machine Learning (ML) breakthroughs in industry and the sciences rely on neural network architectures with multiple layers, abranch of ML called Deep Learning (DL). Over the last decade, much of this progress was possible thanks to the impressive growth ofthe size of datasets and neural networks. In contrast, the understanding of the foundations of DL has not followed this successfultrend. In fact, there is an enormous gap between the practical success of DL and our understanding of why DL works so well. It iswidely acknowledged that to expand the scope of ML applications and obtain reliable artificial intelligence systems, we must achievea fundamental understanding of DL.A physics-based framework can provide a unique perspective to pressing questions in DL and contribute to filling the gap betweenpractical developments and theoretical foundations. We stress that, while ML applications are broadly used in physics, the flow ofideas in the opposite direction, i.e., the use of concepts and techniques from theoretical physics to understand modern deep learning,has only started to be explored.In this MSCA, we will exploit and capitalize on the striking similarities between deep neural networks and effective field theory inphysics. The goal of "Phenomenology of Deep Learning" (PHENO-DL) is to contribute to the development of an effective theory ofdeep learning. We will make use of physics model-building tools to attack foundational questions (e.g., how do deep neural networksgeneralize?) and remarkable phenomena in deep learning (namely, double descent and adversarial examples). To this end, we willadopt a setup inspired by established methods from theoretical physics, which have been recently applied to neural networks. Inparticular, to explain the neural network's expressivity and capacity, we will use the interplay between the renormalisation group inphysics and the 'hierarchy of features' in the different layers in a deep neural network.