ON THE RATE OF CONVERGENCE OF FULLY CONNECTED DEEP NEURAL NETWORK REGRESSION ESTIMATES
ON THE RATE OF CONVERGENCE OF FULLY CONNECTED DEEP NEURAL NETWORK REGRESSION ESTIMATES
复制标题
DOI：
10.1214/20-aos2034
复制
发表时间：
2021-08-01
影响因子：
4.5
通讯作者：
Langer, Sophie
中科院分区：
文献类型：
--
作者：
Kohler, Michael;Langer, Sophie
非参数回归的最新结果表明，深度学习，即具有许多隐藏层的神经网络估计，能够规避所谓的维度诅咒，以防适当地限制回归函数结构的限制。这些结果中使用的神经网络的一个关键特征是它们的网络体系结构具有进一步的约束，即网络稀疏性。在本文中，我们表明，基于简单的完全连接的神经网络具有Relu激活功能，我们也可以获得相似的结果。在这里，要么固定每个隐藏层的神经元的数量，而且隐藏层的数量往往是无限的，对于趋向于无穷大的样本量，或者隐藏层的数量在样本量和样本量中的某些对数因素和数量界定。每个隐藏层的神经元倾向于适当地快速快速，以使样本量趋向于无穷大。该证明是基于有关深神经网络的新近似结果。
Recent results in nonparametric regression show that deep learning, that is, neural network estimates with many hidden layers, are able to circumvent the so-called curse of dimensionality in case that suitable restrictions on the structure of the regression function hold. One key feature of the neural networks used in these results is that their network architecture has a further constraint, namely the network sparsity. In this paper, we show that we can get similar results also for least squares estimates based on simple fully connected neural networks with ReLU activation functions. Here, either the number of neurons per hidden layer is fixed and the number of hidden layers tends to infinity suitably fast for sample size tending to infinity, or the number of hidden layers is bounded by some logarithmic factor in the sample size and the number of neurons per hidden layer tends to infinity suitably fast for sample size tending to infinity. The proof is based on new approximation results concerning deep neural networks.