APPROXIMATION AND ESTIMATION BOUNDS FOR ARTIFICIAL NEURAL NETWORKS
APPROXIMATION AND ESTIMATION BOUNDS FOR ARTIFICIAL NEURAL NETWORKS
复制标题
DOI：
10.1023/a:1022650905902
复制
发表时间：
1994-01-01
期刊：
影响因子：
7.5
通讯作者：
BARRON, AR
中科院分区：
文献类型：
--
作者：
BARRON, AR
对于一类常见的人工神经网络，估计的网络和目标函数f之间的平均积分平方误差被证明是有界的O（C（f）2/n）+ O（nd/NlogN），其中n是节点数，d是函数的输入维数，N是训练观测数，C（f）是f的傅里叶幅度分布的一阶绝对矩。对这个总风险的两个贡献是近似误差和估计误差。近似误差是指目标函数与给定架构的最接近的神经网络函数之间的距离，估计误差是指该理想网络函数与估计的网络函数之间的距离。当n个节点近似为C（f）（N/（d log N））1/2时，积分均方误差的界的阶数优化为O（C（f）（（d/N）log N）1/2）.的界限表明令人惊讶的有利性能的网络估计相比，传统的系列和非参数曲线估计技术的情况下，d是适度大。当节点的数量n不是预先选择为C（f）的函数（其通常不是先验已知的），而是通过使用复杂性正则化或最小描述长度标准从观察到的数据优化节点的数量时，获得类似的界限。分析涉及傅立叶技术的近似误差，度量熵的估计误差的考虑，和计算的指数的可解析性的最小复杂性估计的家庭网络。
For a common class of artificial neural networks, the mean integrated squared error between the estimated network and a target function f is shown to be bounded by O(C(f)2/n) + O(nd/N log N), where n is the number of nodes, d is the input dimension of the function, N is the number of training observations, and C(f) is the first absolute moment of the Fourier magnitude distribution of f. The two contributions to this total risk are the approximation error and the estimation error. Approximation error refers to the distance between the target function and the closest neural network function of a given architecture and estimation error refers to the distance between this ideal network function and an estimated network function. With n approximately C(f) (N/(d log N))1/2 nodes, the order of the bound on the mean integrated squared error is optimized to be O(C(f) ((d/N) log N)1/2). The bound demonstrates surprisingly favorable properties of network estimation compared to traditional series and nonparametric curve estimation techniques in the case that d is moderately large. Similar bounds are obtained when the number of nodes n is not preselected as a function of C(f) (which is generally not known a priori), but rather the number of nodes is optimized from the observed data by the use of a complexity regularization or minimum description length criterion. The analysis involves Fourier techniques for the approximation error, metric entropy considerations for the estimation error, and a calculation of the index of resolvability of minimum complexity estimation of the family of networks.