Relaxed conditions for convergence analysis of online back-propagation algorithm with L2 regularizer for Sigma-Pi-Sigma neural network
Relaxed conditions for convergence analysis of online back-propagation algorithm with L2 regularizer for Sigma-Pi-Sigma neural network
复制标题
Sigma-Pi-Sigma 神经网络 L-2 正则化在线反向传播算法收敛分析的宽松条件
DOI：
10.1016/j.neucom.2017.06.057
复制
发表时间：
2018-01-10
期刊：
影响因子：
6
通讯作者：
Zhang, Chao
中科院分区：
文献类型：
--
作者：
Liu, Yan;Yang, Dakun;Zhang, Chao
在用L-2正则化方法训练Sigma-PI-Sigma神经网络的在线反向传播方法过程中，研究了有界性估计的性质。本文给出了统一的收敛分析，利用了White的随机逼近方法的定理。我们应用正则化方法得到了Sigma-PI-Sigma网络的估计界，并给出了确定收敛的条件，保证了反向传播估计器几乎必然收敛到一个局部最小化期望平方误差损失的参数值。此外，通过平方正则化得到了一些权有界性估计，然后利用有界性证明了算法的收敛。文中还给出了仿真结果，验证了理论分析结果。(C)2017爱思唯尔B.V.保留所有权利。
The properties of a boundedness estimations are investigated during the training of online back-propagation method with L-2 regularizer for Sigma-Pi-Sigma neural network. This brief presents a unified convergence analysis, exploiting theorems of White for the method of stochastic approximation. We apply the method of regularizer to derive estimation bounds for Sigma-Pi-Sigma network, and also give conditions for determinating convergence ensuring that the back-propagation estimator converges almost surely to a parameter value which locally minimizes the expected squared error loss. Besides, some weight boundedness estimations are derived through the squared regularizer, after that the boundedness is exploited to prove the convergence of the algorithm. A simulation is also given to verify the theoretical findings. (C) 2017 Elsevier B.V. All rights reserved.