Relation between weight size and degree of over-fitting in neural network regression
Relation between weight size and degree of over-fitting in neural network regression
复制标题
DOI：
10.1016/j.neunet.2007.11.001
复制
发表时间：
2008-01-01
期刊：
影响因子：
7.8
通讯作者：
Fukunaizu, Kenji
中科院分区：
文献类型：
--
作者：
Hagiwara, Katsuyuki;Fukunaizu, Kenji
本文研究了神经网络回归中的过拟合和权重大小之间的关系。讨论了网络对高斯噪声的过度拟合。使用重新参数化，网络函数表示为有界函数 g 乘以系数 c。这被认为将给定输入处 g 的输出平方和限制在正常数 3 之外，从而限制了网络的权重大小并能够导出过拟合程度的概率上限。这揭示了概率上界的阶数可以根据S而变化，通过应用该下界来分析一个高斯单元的过拟合行为，结果表明，当样本量很大时，在训练中获得宽度参数极小的值的概率接近于1。 (c) 2007 Elsevier Ltd. 保留所有权利。
This paper investigates the relation between over-fitting and weight size in neural network regression. The over-fitting of a network to Gaussian noise is discussed. Using re-parametrization, a network function is represented as a bounded function g multiplied by a coefficient c. This is considered to bound the squared sum of the outputs of g at given inputs away from a positive constant 3,, which restricts the weight size of a network and enables the probabilistic upper bound of the degree of over-fitting to be derived. This reveals that the order of the probabilistic upper bound can change depending on S, By applying the bound to analyze the over-fitting behavior of one Gaussian unit, it is shown that the probability of obtaining an extremely small value for the width parameter in training is close to one when the sample size is large. (c) 2007 Elsevier Ltd. All rights reserved.