Free probability aspects of neural networks

神经网络的自由概率方面

• 批准号: 461815964
• 负责人: Professor Dr. Roland Speicher
• 金额: --
• 依托单位: Arbeitsgruppe Speicher - Freie Wahrscheinlichkeit
• 依托单位国家: 德国
• 项目类别: Research Grants
• 资助国家: 德国
• 项目状态: 未结题
• 来源: https://gepris.dfg.de/gepris/projekt/461815964?language=en
• 关键词: Free; probability; aspects; neural; networks

Machine learning and, in particular, deep neural networks have made tremendous practical progress in the last couple of years, with many successful applications over a wide range of disciplines. However, the mathematical foundation of this success is still not well understood.A neural network, in its mathematical incarnation, is a quite general composition of matrices and entry-wise applied non-linear functions. In the limit of infinite width the matrix part is amenable to methods from random matrix and free probability theory. The inclusion of non-linearities in random matrix considerations is a quite recent challenge, mainly motivated by the relevance for neural networks. In the last few years, there have been approaches relying on random matrix and/or free probability theory to investigate questions around deep learning. The goal of the present project is to streamline and generalize those previous investigations and put them on a more systematic foundation.

机器学习，特别是深度神经网络在过去几年中取得了巨大的实际进展，在广泛的学科中有许多成功的应用。然而，这种成功的数学基础仍然没有得到很好的理解。神经网络，在其数学化身，是一个相当普遍的组合矩阵和条目明智的应用非线性函数。在无限宽的限制下，矩阵部分服从随机矩阵和自由概率论的方法。在随机矩阵考虑中包含非线性是一个相当新的挑战，主要是由神经网络的相关性激发的。在过去的几年里，出现了一些依赖随机矩阵和/或自由概率理论来研究深度学习问题的方法。本项目的目标是精简和概括以前的调查，并将其置于更系统的基础上。