Random Neural Networks in the Infinite Width Limit as Gaussian Processes
Random Neural Networks in the Infinite Width Limit as Gaussian Processes
复制标题
DOI：
10.1214/23-aap1933
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
B. Hanin
中科院分区：
文献类型：
--
作者：
B. Hanin
本文给出了一个新的证明，在输入、输出和深度保持不变，而隐含层宽度趋于无穷大的情况下，具有随机权和偏差的全连接神经网络收敛于高斯过程。与以前的工作不同的是，对于权重分布和相当一般的非线性，仅假设矩条件下的收敛。
This article gives a new proof that fully connected neural networks with random weights and biases converge to Gaussian processes in the regime where the input dimension, output dimension, and depth are kept fixed, while the hidden layer widths tend to infinity. Unlike prior work, convergence is shown assuming only moment conditions for the distribution of weights and for quite general non-linearities.