Should backpropagation be replaced by more effective optimization algorithms?
Should backpropagation be replaced by more effective optimization algorithms?
复制标题
反向传播是否应该被更有效的优化算法取代？
DOI：
10.1109/ijcnn.1991.155202
复制
发表时间：
1991
期刊：
影响因子：
--
通讯作者：
D. Himmelblau
中科院分区：
文献类型：
--
作者：
J. Hsiung;W. Suewatanakul;D. Himmelblau
作者建议使用反向传播（BP）作为优化人工神经网络中的权重值的首选技术。他们比较了通过BP和连续二次编程代码的函数表示，后者在实现相同的容错性方面至少快四倍。拟议战略有两个主要特点。一个是它忘记了从输出层到输入层依次调整权重，而是一次调整整个权重集。第二个特征是它在迭代的一个阶段通过网络传递整个模式集，并使用所有模式的所有误差的平方和作为目标函数。该策略的另一个特点是，它使用了一个非线性优化代码，适应约束条件，如广义简约梯度法或连续二次规划，以调整所有的权重和其他参数。<<ETX>>
The authors propose the use of backpropagation (BP) as the preferred technique of optimizing the values of the weights in an artificial neural network. They compare functional representation via BP and a successive quadratic programming code, with the latter being at least four times faster in achieving the same error tolerance. The proposed strategy has two main features. One is that it forgets about adjusting the weights sequentially from the output layer to the input layer, and instead adjusts the entire set of weights at once. The second feature is that it passes the entire set of patterns through the network on one stage of iteration and uses the sum of the squares of all of the errors for all the patterns as the objective function. Another feature of the strategy is that it uses a nonlinear optimization code that accommodates constraints, such as the generalized reduced gradient method or successive quadratic programming, to adjust all the weights and other parameters.<<ETX>>