An Approach to Stable Gradient-Descent Adaptation of Higher Order Neural Units
An Approach to Stable Gradient-Descent Adaptation of Higher Order Neural Units
复制标题
DOI：
10.1109/tnnls.2016.2572310
复制
发表时间：
2017-09-01
影响因子：
10.4
通讯作者：
Homma, Noriyasu
中科院分区：
文献类型：
--
作者：
Bukovsky, Ivo;Homma, Noriyasu
介绍了一种利用梯度下降法对高阶神经元（HONUs）的权值更新系统进行稳定性评估的方法，该系统具有神经元输入的多项式聚集（也称为多项式神经网络类），用于前馈和递归HONUs的自适应.该方法的一个基本核心是基于权重更新系统的谱半径，并且它允许在每个适应步骤单独地进行稳定性监测和维护。确保权重更新系统的稳定性（在每个单个自适应步骤）自然会导致整个神经架构适应目标数据的自适应稳定性。另外，所使用的方法强调了HONU的权重优化是线性问题的事实，因此所提出的方法通常可以扩展到其自适应参数为线性的任何神经架构。
Stability evaluation of a weight-update system of higher order neural units (HONUs) with polynomial aggregation of neural inputs (also known as classes of polynomial neural networks) for adaptation of both feedforward and recurrent HONUs by a gradient descent method is introduced. An essential core of the approach is based on the spectral radius of a weight-update system, and it allows stability monitoring and its maintenance at every adaptation step individually. Assuring the stability of the weight-update system (at every single adaptation step) naturally results in the adaptation stability of the whole neural architecture that adapts to the target data. As an aside, the used approach highlights the fact that the weight optimization of HONU is a linear problem, so the proposed approach can be generally extended to any neural architecture that is linear in its adaptable parameters.