Regularized online sequential learning algorithm for single-hidden layer feedforward neural networks
Regularized online sequential learning algorithm for single-hidden layer feedforward neural networks
复制标题
DOI：
10.1016/j.patrec.2011.07.016
复制
发表时间：
2011-10-15
影响因子：
5.1
通讯作者：
Won, Yonggwan
中科院分区：
文献类型：
--
作者：
Hieu Trung Huynh;Won, Yonggwan
由于能够通过顺序到达的数据进行学习，在线学习算法在许多应用中受到青睐。最近提出的用于训练单隐层前馈神经网络（SLFNs）的有效算法之一是在线序贯极限学习机（OS - ELM），它可以逐个或逐块地学习固定或不同大小的数据。它基于极限学习机（ELM）的思想，即随机选择输入权重和隐层偏置，然后通过伪逆运算确定输出权重。该算法的学习速度极快。然而，它对于有噪声的数据难以生成泛化模型，并且难以初始化参数以避免奇异和不适定问题。在本文中，我们基于双目标优化方法提出了一种对OS - ELM的改进。它试图最小化经验误差并获得网络权重向量的小范数。通过使用蒂霍诺夫正则化可以克服奇异和不适定问题。这种方法也能够逐个或逐块地学习数据。实验结果表明，所提出的方法在基准数据集上具有更好的泛化性能。© 2011爱思唯尔B.V. 保留所有权利。
Online learning algorithms have been preferred in many applications due to their ability to learn by the sequentially arriving data. One of the effective algorithms recently proposed for training single hidden-layer feedforward neural networks (SLFNs) is online sequential extreme learning machine (OS-ELM), which can learn data one-by-one or chunk-by-chunk at fixed or varying sizes. It is based on the ideas of extreme learning machine (ELM), in which the input weights and hidden layer biases are randomly chosen and then the output weights are determined by the pseudo-inverse operation. The learning speed of this algorithm is extremely high. However, it is not good to yield generalization models for noisy data and is difficult to initialize parameters in order to avoid singular and ill-posed problems. In this paper, we propose an improvement of OS-ELM based on the bi-objective optimization approach. It tries to minimize the empirical error and obtain small norm of network weight vector. Singular and ill-posed problems can be overcome by using the Tikhonov regularization. This approach is also able to learn data one-by-one or chunk-by-chunk. Experimental results show the better generalization performance of the proposed approach on benchmark datasets. (C) 2011 Elsevier B.V. All rights reserved.