Adaptive method of realizing natural gradient learning for multilayer perceptrons
Adaptive method of realizing natural gradient learning for multilayer perceptrons
复制标题
DOI：
10.1162/089976600300015420
复制
发表时间：
2000-06-01
影响因子：
2.9
通讯作者：
Fukumizu, K
中科院分区：
文献类型：
--
作者：
Amari, S;Park, H;Fukumizu, K
自然梯度学习方法是一种在线训练多层感知器的理想方法。它避免了高原，这会导致反向传播方法收敛缓慢。它是Fisher有效的，而传统的方法不是。然而，为了实现该方法，需要计算Fisher信息矩阵及其逆，这在实际中是非常困难的。本文提出了一种直接求Fisher信息矩阵逆的自适应方法。它推广了自适应高斯-牛顿算法，并提供了坚实的理论证明。仿真结果表明，所提出的自适应方法可以很好地实现自然梯度学习。
The natural gradient learning method is known to have ideal performances for on-line training of multilayer perceptrons. It avoids plateaus, which give rise to slow convergence of the backpropagation method. It is Fisher efficient, whereas the conventional method is not. However, for implementing the method, it is necessary to calculate the Fisher information matrix and its inverse, which is practically very difficult. This article proposes an adaptive method of directly obtaining the inverse of the Fisher information matrix. It generalizes the adaptive Gauss-Newton algorithms and provides a solid theoretical justification of them. Simulations show that the proposed adaptive method works very well for realizing natural gradient learning.