A fast new algorithm for training feedforward neural networks
A fast new algorithm for training feedforward neural networks
复制标题
一种用于训练前馈神经网络的快速新算法
DOI：
10.1109/78.157194
复制
发表时间：
1992
期刊：
影响因子：
--
通讯作者：
N. Tepedelenlioglu
中科院分区：
文献类型：
--
作者：
Robert S. Scalero;N. Tepedelenlioglu
提出了一种快速训练多层感知器的新算法，作为反向传播算法的替代算法。该算法大大缩短了训练时间，克服了传统反向传播算法存在的许多缺点。
A fast new algorithm is presented for training multilayer perceptrons as an alternative to the back-propagation algorithm. This new algorithm reduces the required training time considerably and overcomes many of the shortcomings presented by the conventional back-propagation algorithm.
The new algorithm shortens the training time by several orders of magnitude for the pattern recognition type considered. In some cases improvement ratios of the new algorithm over the back-propagation algorithm run higher than 10,000, and it is not unlikely that this number may be further increased by considering patterns with higher resolution (more pixels per pattern).
The new algorithm can also be implemented in a parallel architecture. One processor handles the network processing and the calculation of the modified back-propagation error signals, while the other processor handles the Kalman filter calculations.
Aside from the speed advantage, the new algorithm is also more predictable in its training. The algorithm makes steady progress toward improving the mean squared error. In contrast, the back-propagation algorithm tends to reach a certain mean squared error and remain there for many iterations making little or no progress. At some point, it either rapidly converges, or jumps to a new level where it would again make little or no progress for many iterations.
The convergence of the back-propagation algorithm depends heavily on the magnitude of the initial weights. If chosen incorrectly, the algorithm takes a long time to converge. The new algorithm on the other hand is much less sensitive to the initial weight setting.
Furthermore, the adaptive nature of the Kalman gain makes the new algorithm much less likely to get caught in a state other than the global minimum.
The new algorithm is much faster and more reliable than the back-propagation algorithm. It is very consistent in its training and is much less sensitive to the initial weight settings than the back-propagation algorithm. In every aspect the new algorithm outperforms the back-propagation algorithm for training multilayer perceptrons. (Abstract shortened with permission of author.)