Improved Computation for Levenberg-Marquardt Training
Improved Computation for Levenberg-Marquardt Training
复制标题
DOI：
10.1109/tnn.2010.2045657
复制
发表时间：
2010-06-01
影响因子：
--
通讯作者：
Yu, Hao
中科院分区：
文献类型：
--
作者：
Wilamowski, Bogdan M.;Yu, Hao
提出了一种改进的算法，旨在优化使用LM算法的神经网络学习过程。该算法无需雅可比矩阵的乘法运算和存储，直接计算拟海森矩阵和梯度向量。解决了LM训练的内存限制问题。考虑到拟Hessian矩阵的对称性，只需计算其上/下三角阵中的元素。因此，训练速度显着提高，不仅是因为存储在内存中的数组较小，而且还减少了准Hessian矩阵计算中的操作。改进的记忆和时间效率对于大尺寸模式训练尤其如此。
The improved computation presented in this paper is aimed to optimize the neural networks learning process using Levenberg-Marquardt (LM) algorithm. Quasi-Hessian matrix and gradient vector are computed directly, without Jacobian matrix multiplication and storage. The memory limitation problem for LM training is solved. Considering the symmetry of quasi-Hessian matrix, only elements in its upper/lower triangular array need to be calculated. Therefore, training speed is improved significantly, not only because of the smaller array stored in memory, but also the reduced operations in quasi-Hessian matrix calculation. The improved memory and time efficiencies are especially true for large sized patterns training.