TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization
TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization
复制标题
DOI：
10.1016/j.neucom.2010.12.042
复制
发表时间：
2011-09-01
期刊：
影响因子：
6
通讯作者：
Lendasse, Amaury
中科院分区：
文献类型：
--
作者：
Miche, Yoan;van Heeswijk, Mark;Lendasse, Amaury
本文提出了一种改进的最优修剪极端学习机（OP-ELM）的形式的L-2正则化惩罚内的OP-ELM。OP-ELM最初提出了一种围绕极端学习机（ELM）的包装方法，旨在降低ELM对不相关变量的敏感性，并通过神经元修剪获得更简约的模型。OP-ELM的拟议修改使用两个正则化惩罚的级联：首先是L-1惩罚来对隐藏层的神经元进行排名，然后是回归权重（隐藏层和输出层之间的回归）的L-2惩罚，以实现数值稳定性和有效的神经元修剪。新方法在11个不同的数据集上与最先进的方法（如支持向量机或高斯过程和原始ELM和OP-ELM）进行了测试;它系统地优于OP-ELM（平均均方误差提高27%），并在结果的标准差方面提供了更可靠的结果-同时始终比OP-ELM慢不到一个数量级。(C)2011爱思唯尔有限公司版权所有。
In this paper an improvement of the optimally pruned extreme learning machine (OP-ELM) in the form of a L-2 regularization penalty applied within the OP-ELM is proposed. The OP-ELM originally proposes a wrapper methodology around the extreme learning machine (ELM) meant to reduce the sensitivity of the ELM to irrelevant variables and obtain more parsimonious models thanks to neuron pruning. The proposed modification of the OP-ELM uses a cascade of two regularization penalties: first a L-1 penalty to rank the neurons of the hidden layer, followed by a L-2 penalty on the regression weights (regression between hidden layer and output layer) for numerical stability and efficient pruning of the neurons. The new methodology is tested against state of the art methods such as support vector machines or Gaussian processes and the original ELM and OP-ELM, on 11 different data sets; it systematically outperforms the OP-ELM (average of 27% better mean square error) and provides more reliable results in terms of standard deviation of the results - while remaining always less than one order of magnitude slower than the OP-ELM. (C) 2011 Elsevier B.V. All rights reserved.