Efficient agnostic learning of neural networks with bounded fan-in
Efficient agnostic learning of neural networks with bounded fan-in
复制标题
DOI：
10.1109/18.556601
复制
发表时间：
1996-11-01
影响因子：
2.5
通讯作者：
Williamson, RC
中科院分区：
文献类型：
--
作者：
Lee, WS;Bartlett, PL;Williamson, RC
我们证明，具有有界扇入的两层神经网络类在对大概近似正确（PAC）学习模型的现实扩展中是可以有效学习的，在该模型中，假设观测值上存在联合概率分布，并且学习器需要逼近神经网络，从而最小化预期的二次误差。作为特殊情况，该模型允许学习具有有界噪声的实值函数，学习概率概念，并学习神经网络无法很好逼近的目标函数的最佳逼近。网络，我们考虑的网络具有实值输入和输出，具有有限扇入的无限数量的阈值隐藏单元，以及输出权重绝对值总和的界限，学习算法的计算步骤数受 1/epsilon、1/delta、n 和 B 中的多项式限制，其中 epsilon 是所需的精度，delta 是算法失败的概率，n 是输入维度，B 是目标绝对值的界限（可能是随机变量）和输出权重的绝对值之和，在获得结果的过程中，我们还扩展了函数类凸包闭包中函数的迭代逼近以及二次损失函数的不可知学习的样本复杂性的一些结果。
We show that the class of two-layer neural networks with bounded fan-in is efficiently learnable in a realistic extension to the Probably Approximately Correct (PAC) learning model, In this model, a joint probability distribution is assumed to exist on the observations and the learner is required to approximate the neural network which minimizes the expected quadratic error, As special cases, the model allows learning real-valued functions with bounded noise, learning probabilistic concepts, and learning the best approximation to a target function that cannot be well approximated by the neural network, The networks we consider have real-valued inputs and outputs, an unlimited number of threshold hidden units with bounded fan-in, and a bound on the sum of the absolute values of the output weights, The number of computation steps of the learning algorithm is bounded by a polynomial in 1/epsilon, 1/delta, n and B where epsilon is the desired accuracy, delta is the probability that the algorithm fails, n is the input dimension, and B is the bound on both the absolute value of the target (which may be a random variable) and the sum of the absolute values of the output weights, In obtaining the result, we also extended some results on iterative approximation of functions in the closure of the convex hull of a function class and on the sample complexity of agnostic learning with the quadratic loss function.