A fast and accurate online sequential learning algorithm for feedforward networks
A fast and accurate online sequential learning algorithm for feedforward networks
复制标题
DOI：
10.1109/tnn.2006.880583
复制
发表时间：
2006-11-01
影响因子：
--
通讯作者：
Sundararajan, N.
中科院分区：
文献类型：
--
作者：
Liang, Nan-Ying;Huang, Guang-Bin;Sundararajan, N.
本文提出了一种单隐层前馈网络的在线序贯学习算法。在统一框架中添加或径向基函数（RBF）隐藏节点。该算法被称为在线顺序极端学习机（OS-ELM），可以逐个或逐块（一块数据）学习数据，具有固定或变化的块大小。OS-ELM中可加节点的激活函数可以是任意有界非常数分段连续函数，RBF节点的激活函数可以是任意可积分段连续函数。在OS-ELM中，隐节点的参数（加性节点的输入权重和偏差或RBF节点的中心和影响因子）是随机选择的，输出权重是基于顺序到达的数据解析确定的。该算法使用了Huang等人的ELM的思想。开发用于批量学习的ELM已被证明是非常快的，泛化性能优于其他批量训练方法。除了选择隐藏节点的数量外，无需手动选择其他控制参数。详细的性能比较OS-ELM与其他流行的顺序学习算法的基准问题，从回归，分类和时间序列预测领域。实验结果表明，OS-ELM算法比其他序列算法具有更快的速度和更好的泛化性能。
In this paper, we develop an online sequential learning algorithm for single hidden layer feedforward networks (SLFNs) with. additive or radial basis function (RBF) hidden nodes in a unified framework. The algorithm is referred to as online sequential extreme learning machine (OS-ELM) and can learn data one-by-one or chunk-by-chunk (a block of data) with fixed or varying chunk size. The activation functions for additive nodes in OS-ELM can be any bounded nonconstant piecewise continuous functions and the activation functions for RBF nodes can be any integrable piecewise continuous functions. In OS-ELM, the parameters of hidden nodes (the input weights and biases of additive nodes or the centers and impact factors of RBF nodes) are randomly selected and the output weights are analytically determined based on the sequentially arriving data. The algorithm uses the ideas of ELM of Huang et al. developed for batch learning which has been shown to be extremely fast with generalization performance better than other batch training methods. Apart from selecting the number of hidden nodes, no other control parameters have to be manually chosen. Detailed performance comparison of OS-ELM is done with other popular sequential learning algorithms on benchmark problems drawn from the regression, classification and time series prediction areas. The results show that the OS-ELM is faster than the other sequential algorithms and produces better generalization performance.