Neural networks and quantum field theory
Neural networks and quantum field theory
复制标题
DOI：
10.1088/2632-2153/abeca3
复制
发表时间：
2021-09-01
影响因子：
6.8
通讯作者：
Stoner, Keegan
中科院分区：
文献类型：
--
作者：
Halverson, James;Maiti, Anindita;Stoner, Keegan
根据威尔逊有效场理论，我们提出了对神经网络的理论理解。这种对应关系依赖于这样一个事实，即许多渐近神经网络是从高斯过程(GP)中提取的，高斯过程是非相互作用场理论的类似。远离渐近极限会产生非高斯过程(NGP)，并对应于开启粒子相互作用，从而允许使用Feynman图计算神经网络输出的关联函数。最小的NGP似然是由最相关的非高斯项决定的，根据威尔逊重整化群诱导的系数流。这在过度参数化和神经网络可能性的简单性之间产生了直接的联系。无论系数是常量还是函数，都可以用GP极限对称性来理解，正如t Hooft的技术自然性所期望的那样。在允许对应的最简单的模型类中，一般的理论计算与神经网络实验相匹配。我们的形式对于在渐近极限中成为GP的许多体系结构中的任何一个都有效，这一性质在某些类型的训练下得到保留。
We propose a theoretical understanding of neural networks in terms of Wilsonian effective field theory. The correspondence relies on the fact that many asymptotic neural networks are drawn from Gaussian processes (GPs), the analog of non-interacting field theories. Moving away from the asymptotic limit yields a non-Gaussian process (NGP) and corresponds to turning on particle interactions, allowing for the computation of correlation functions of neural network outputs with Feynman diagrams. Minimal NGP likelihoods are determined by the most relevant non-Gaussian terms, according to the flow in their coefficients induced by the Wilsonian renormalization group. This yields a direct connection between overparameterization and simplicity of neural network likelihoods. Whether the coefficients are constants or functions may be understood in terms of GP limit symmetries, as expected from 't Hooft's technical naturalness. General theoretical calculations are matched to neural network experiments in the simplest class of models allowing the correspondence. Our formalism is valid for any of the many architectures that becomes a GP in an asymptotic limit, a property preserved under certain types of training.