A comparative analysis of support vector machines and extreme learning machines
A comparative analysis of support vector machines and extreme learning machines
复制标题
支持向量机与极限学习机的对比分析
DOI：
10.1016/j.neunet.2012.04.002
复制
发表时间：
2012-09-01
期刊：
影响因子：
7.8
通讯作者：
Li, Ping
中科院分区：
文献类型：
--
作者：
Liu, Xueyi;Gao, Chuanhou;Li, Ping
极限学习机(ELMS)理论最近变得越来越流行。ELM作为一种新的单隐层前馈神经网络学习算法，具有计算量小、泛化能力强、易于实现等优点。因此，ELMS与其他最先进的机器学习方法的比较和模型选择就变得非常重要，并吸引了许多研究工作。本文从Vapnik-Chervonenkis(Vapnik-Chervonenkis，VC)维度和不同训练样本量下的性能两个不同的角度对基本ELMS和支持向量机(SVMs)进行了比较分析。证明了ELM的VC维等于ELM的隐节点数，概率为1。此外，随着训练样本大小的变化，它们的泛化能力和计算复杂性也得到了体现。ELMS在小样本情况下的泛化能力弱于支持向量机，但在大样本情况下具有与支持向量机相同的泛化能力。值得注意的是，ELMS在计算速度上有很大的优势，特别是在处理大规模样本问题时。所获得的结果可以洞察它们之间的本质关系，也可以作为他们过去的实验和理论比较的补充知识。(C)2012爱思唯尔有限公司。保留所有权利。
The theory of extreme learning machines (ELMs) has recently become increasingly popular. As a new learning algorithm for single-hidden-layer feed-forward neural networks, an ELM offers the advantages of low computational cost, good generalization ability, and ease of implementation. Hence the comparison and model selection between ELMs and other kinds of state-of-the-art machine learning approaches has become significant and has attracted many research efforts. This paper performs a comparative analysis of the basic ELMs and support vector machines (SVMs) from two viewpoints that are different from previous works: one is the Vapnik-Chervonenkis (VC) dimension, and the other is their performance under different training sample sizes. It is shown that the VC dimension of an ELM is equal to the number of hidden nodes of the ELM with probability one. Additionally, their generalization ability and computational complexity are exhibited with changing training sample size. ELMs have weaker generalization ability than SVMs for small sample but can generalize as well as SVMs for large sample. Remarkably, great superiority in computational speed especially for large-scale sample problems is found in ELMs. The results obtained can provide insight into the essential relationship between them, and can also serve as complementary knowledge for their past experimental and theoretical comparisons. (c) 2012 Elsevier Ltd. All rights reserved.