On-line learning from finite training sets
On-line learning from finite training sets
复制标题
从有限训练集在线学习
DOI：
10.1209/epl/i1997-00271-3
复制
发表时间：
1997
期刊：
影响因子：
--
通讯作者：
D. Barber
中科院分区：
文献类型：
--
作者：
Peter Sollich;D. Barber
我们分析了以非无穷小学习率η从有限的训练样本集中在线（梯度下降）学习规则，精确计算了简单模型场景的时间相关泛化误差。在热力学极限下，我们用“样本内自平均”方法封闭了无穷阶序参量生成函数的动力学方程。由此产生的动力学在η中是非微扰的，慢模式仅在有限阈值ηmin以上出现。对于给定的最终学习时间，确定η的最佳设置，并将结果与离线梯度下降进行比较。
We analyse on-line (gradient descent) learning of a rule from a finite set of training examples at non-infinitesimal learning rates η, calculating exactly the time-dependent generalization error for a simple model scenario. In the thermodynamic limit, we close the dynamical equation for the generating function of an infinite hierarchy of order parameters using “within-sample self-averaging”. The resulting dynamics is non-perturbative in η, with a slow mode appearing only above a finite threshold ηmin. Optimal settings of η for given final learning time are determined and the results are compared with offline gradient descent.