Transient dynamics of on-line learning in two-layered neural networks
Transient dynamics of on-line learning in two-layered neural networks
复制标题
两层神经网络在线学习的瞬态动力学
DOI：
--
复制
发表时间：
1996
期刊：
影响因子：
--
通讯作者：
C. Wöhler
中科院分区：
文献类型：
--
作者：
Michael Biehl;P. Riegler;C. Wöhler
在具有连续单元的神经网络中，在线学习的动力学在泛化误差的时间依赖性中由平台主导。使用统计力学的工具，我们证明了软委员会机器存在几个不动点的动态学习，产生复杂的行为，如级联运行通过不同的高原与相应的泛化误差的值递减。我们发现学习率依赖的现象，如分裂和消失的不动点的运动方程。分析了平台长度对初始条件的依赖关系，模拟结果证实了这一点。
The dynamics of on-line learning in neural networks with continuous units is dominated by plateaux in the time dependence of the generalization error. Using tools from statistical mechanics, we show for a soft committee machine the existence of several fixed points of the dynamics of learning that give rise to complicated behaviour, such as cascade- like runs through different plateaux with a decreasing value of the corresponding generalization error. We find learning-rate-dependent phenomena, such as splitting and disappearing of fixed points of the equations of motion. The dependence of plateau lengths on the initial conditions is described analytically and simulations confirm the results.