Distributed Extreme Learning Machine with kernels based on MapReduce
Distributed Extreme Learning Machine with kernels based on MapReduce
复制标题
基于MapReduce内核的分布式极限学习机
DOI：
10.1016/j.neucom.2014.01.070
复制
发表时间：
2015-02-03
期刊：
影响因子：
6
通讯作者：
Wang, Chao
中科院分区：
文献类型：
--
作者：
Bi, Xin;Zhao, Xiangguo;Wang, Chao
Extreme Learning Machine（ELM）在许多学习应用中显示出良好的概括性能和非常快的学习速度。最近，已经证明，从优化的角度来看，ELM的表现优于支持向量机（SVM）。 ELM提供具有广泛类型的功能映射类型的统一学习方案。在这些统一的算法中，具有内核的ELM应用核代替随机特征映射。但是，随着大规模学习应用中培训数据的数量呈指数增长，带有内核的集中型ELM遭受了大型矩阵操作的大量记忆消耗。此外，由于沟通成本的高，这些矩阵操作中的某些无法直接在共享的无分布式计算模型（如MapReduce）上实施。本文提出了一个名为分布式内核ELM（DK-ELM）的分布式解决方案，该解决方案实现了用MapReduce上核的ELM实现。还应用了分布式内核矩阵计算和矩阵与矢量的乘法，以实现DK-ELM的并行计算。进行大量数据集的广泛实验，以验证DK-ELM的可伸缩性和训练性能。实验结果表明，DK-ELM对大规模学习应用具有良好的可扩展性。 （c）2014 Elsevier B.V.保留所有权利。
Extreme Learning Machine (ELM) has shown its good generalization performance and extremely fast learning speed in many learning applications. Recently, it has been proved that ELM outperforms Support Vector Machine (SVM) with less constraints from the optimization point of view. ELM provides unified learning schemes with a widespread type of feature mappings. Among these unified algorithms, ELM with kernels applies kernels instead of random feature mappings. However, with the exponentially increasing volume of training data in massive learning applications, centralized ELM with kernels suffers from the great memory consumption of large matrix operations. Besides, due to the high communication cost, some of these matrix operations cannot be directly implemented on shared-nothing distributed computing model like MapReduce. This paper proposes a distributed solution named Distributed Kernelized ELM (DK-ELM), which realizes an implementation of ELM with kernels on MapReduce. Distributed kernel matrix calculation and multiplication of matrix with vector are also applied to realize parallel calculation of DK-ELM. Extensive experiments on massive datasets are conducted to verify both the scalability and training performance of DK-ELM. Experimental results show that DK-ELM has good scalability for massive learning applications. (C) 2014 Elsevier B.V. All rights reserved.