Efficient backprop
Efficient backprop
复制标题
DOI：
10.1007/3-540-49430-8_2
复制
发表时间：
1998-01-01
期刊：
影响因子：
--
通讯作者：
Müller, KR
中科院分区：
文献类型：
--
作者：
LeCun, Y;Bottou, L;Müller, KR
分析了反向传播学习的收敛性，以解释从业者观察到的常见现象。许多不受欢迎的反向操作行为可以通过一些技巧来避免，这些技巧很少在严肃的技术出版物中公开。本文给出了其中的一些技巧，并解释了它们为什么有效。许多作者提出二阶优化方法有利于神经网络的训练。结果表明，大多数“经典”二阶方法对于大型神经网络是不切实际的。提出了几种不受这些限制的方法。
The convergence of back-propagation learning is analyzed so as to explain common phenomenon observed by practitioners. Many undesirable behaviors of backprop can be avoided with tricks that are rarely exposed in serious technical publications. This paper gives some of those tricks, and offers explanations of why they work.Many authors have suggested that second-order optimization methods are advantageous for neural net training. It is shown that most "classical" second-order methods are impractical for large neural networks. A few methods are proposed that do not have these limitations.