Free probability aspects of neural networks

点击翻译按钮获取中文摘要

Machine learning and, in particular, deep neural networks have made tremendous practical progress in the last couple of years, with many successful applications over a wide range of disciplines. However, the mathematical foundation of this success is still not well understood.A neural network, in its mathematical incarnation, is a quite general composition of matrices and entry-wise applied non-linear functions. In the limit of infinite width the matrix part is amenable to methods from random matrix and free probability theory. The inclusion of non-linearities in random matrix considerations is a quite recent challenge, mainly motivated by the relevance for neural networks. In the last few years, there have been approaches relying on random matrix and/or free probability theory to investigate questions around deep learning. The goal of the present project is to streamline and generalize those previous investigations and put them on a more systematic foundation.