Two-layer neural networks with values in a Banach space
Two-layer neural networks with values in a Banach space
复制标题
DOI：
10.1137/21m1458144
复制
发表时间：
2021-05
期刊：
影响因子：
--
通讯作者：
Yury Korolev
中科院分区：
文献类型：
--
作者：
Yury Korolev
本文研究了两层神经网络，其域和值域都是具有可分预测的Banach空间。此外，我们假设像空间具有偏序，即它是Riesz空间。作为非线性，我们选择取正部分的格运算;在R d值神经网络的情况下，这对应于ReLU激活函数。我们证明了逆和直接逼近定理与蒙特-卡罗率的某类功能，扩展现有的结果为有限维的情况下。在论文的第二部分中，我们从正则化理论的角度研究了通过有限数量的噪声观测在潜在空间上通过符号测度找到此类函数的最佳表示的问题。我们讨论的正则性条件被称为源条件，并获得收敛速度在布雷格曼距离为代表的措施在该制度时，噪声水平为零，样本的数量以适当的速率趋于无穷大。
We study two-layer neural networks whose domain and range are Banach spaces with separable preduals. In addition, we assume that the image space is equipped with a partial order, i.e. it is a Riesz space. As the nonlinearity we choose the lattice operation of taking the positive part; in case of R d -valued neural networks this corresponds to the ReLU activation function. We prove inverse and direct approximation theorems with Monte-Carlo rates for a certain class of functions, extending existing results for the finite-dimensional case. In the second part of the paper, we study, from the regularisation theory viewpoint, the problem of finding optimal representations of such functions via signed measures on a latent space from a finite number of noisy observations. We discuss regularity conditions known as source conditions and obtain convergence rates in a Bregman distance for the representing measure in the regime when both the noise level goes to zero and the number of samples goes to infinity at appropriate rates.