Incremental regularized extreme learning machine and it's enhancement
Incremental regularized extreme learning machine and it's enhancement
复制标题
DOI：
10.1016/j.neucom.2015.01.097
复制
发表时间：
2016-01
期刊：
影响因子：
6
通讯作者：
Zhixin Xu;Min Yao;Zhaohui Wu;Weihui Dai
中科院分区：
文献类型：
--
作者：
Zhixin Xu;Min Yao;Zhaohui Wu;Weihui Dai
极限学习机（ELM）是Huang等人提出的一种新的单隐层前向神经网络（SLFN）算法，具有极快的学习速度和良好的泛化性能。当新的隐藏节点被添加到现有网络时，重新训练网络将是耗时的，并且EM-ELM [13]被提出来递增地计算输出权重。然而，EM-ELM仍然存在两个问题：第一，初始隐层输出矩阵可能是秩亏的，从而计算将损失精度;第二，EM-ELM不能总是得到良好的泛化性能，由于过拟合。因此，我们提出了基于正则化方法的EM-ELM的改进版本，称为增量正则化极端学习机（IR-ELM）。当新的隐节点被逐个添加时，IR-ELM能够以快速的方式递归地更新输出权重。本文还介绍了增强的IR-ELM（EIR-ELM），有一个选择的隐藏节点添加到网络中。在回归和分类问题的基准数据集上的实证研究表明，在训练时间相同的情况下，IR-ELM（EIR-ELM）总是比EM-ELM获得更好的泛化性能。
Extreme Learning Machine (ELM) proposed by Huang et al. [2] is a novel algorithm for single hidden layer feedforward neural networks (SLFNs) with extremely fast learning speed and good generalization performance. When new hidden nodes are added to the existing network, retraining the network would be time consuming, and EM-ELM [13] was proposed to calculate the output weights incrementally. However there are still two issues in EM-ELM: first, the initial hidden layer output matrix may be rank deficient thus the computation will loss accuracy; second, EM-ELM cannot always get good generalization performance due to overfitting. So we propose the improved version of EM-ELM based on regularization method called Incremental Regularized Extreme Learning Machine (IR-ELM). When new hidden node is added one by one, IR-ELM can update output weights recursively in a fast way. Enhancement of IR-ELM (EIR-ELM) that has a selection of hidden nodes to be added to the network is also introduced in this paper. Empirical studies on benchmark data sets for regression and classification problems have shown that IR-ELM (EIR-ELM) always gets better generalization performance than EM-ELM with the similar training time.