Barron Spaces and the Compositional Function Spaces for Neural Network Models
Barron Spaces and the Compositional Function Spaces for Neural Network Models
复制标题
DOI：
--
复制
发表时间：
2019-06
期刊：
影响因子：
--
通讯作者：
E. Weinan;Chao Ma;Lei Wu
中科院分区：
文献类型：
--
作者：
E. Weinan;Chao Ma;Lei Wu
分析机器学习模型的关键问题之一是为模型确定合适的函数空间。这是特定机器学习模型可以以良好的精度近似的函数空间，被赋予与近似过程相关联的自然范数。在本文中，我们解决这个问题的两个代表性的神经网络模型：两层网络和残差神经网络。我们定义了巴伦空间，并证明了它是双层神经网络模型的正确空间，在这个意义上，最佳的直接和逆逼近定理在巴伦空间中的函数。对于残差神经网络模型，我们构造了所谓的组合函数空间，并证明了该空间的正逼近定理和逆逼近定理。此外，我们证明了Rademacher复杂度在这些空间中具有最优上界。
One of the key issues in the analysis of machine learning models is to identify the appropriate function space for the model. This is the space of functions that the particular machine learning model can approximate with good accuracy, endowed with a natural norm associated with the approximation process. In this paper, we address this issue for two representative neural network models: the two-layer networks and the residual neural networks. We define Barron space and show that it is the right space for two-layer neural network models in the sense that optimal direct and inverse approximation theorems hold for functions in the Barron space. For residual neural network models, we construct the so-called compositional function space, and prove direct and inverse approximation theorems for this space. In addition, we show that the Rademacher complexity has the optimal upper bounds for these spaces.