Convergence of Gradient Descent for Minimum Error Entropy Principle in Linear Regression
Convergence of Gradient Descent for Minimum Error Entropy Principle in Linear Regression
复制标题
线性回归中最小误差熵原理的梯度下降收敛
DOI：
10.1109/tsp.2016.2612169
复制
发表时间：
2016-12-15
影响因子：
5.4
通讯作者：
Zhou, Ding-Xuan
中科院分区：
文献类型：
--
作者：
Hu, Ting;Wu, Qiang;Zhou, Ding-Xuan
研究了最小误差熵算法在梯度下降下的收敛性。这种方法在实际应用中已经使用了十多年，但一直没有一致性或严格的误差分析。本文首次给出了线性回归环境下解的梯度下降法收敛性的严格证明。证明了均方误差随迭代步数呈指数快速衰减，随样本量m呈O（1）阶衰减。当步长选择适当且尺度参数足够大时，可以保证算法的均方收敛性。
We study the convergence of minimum error entropy (MEE) algorithms when they are implemented by gradient descent. This method has been used in practical applications for more than one decade, but there has been no consistency or rigorous error analysis. This paper gives the first rigorous proof for the convergence of the gradient descent method for MEE in a linear regression setting. The mean square error is proved to decay exponentially fast in terms of the iteration steps and of order O( 1) in terms of the sample size m. The mean square convergence is guaranteed when the step size is chosen appropriately and the scaling parameter is large enough.