Map-Reduce for Machine Learning on Multicore
Map-Reduce for Machine Learning on Multicore
复制标题
DOI：
10.7551/mitpress/7503.003.0040
复制
发表时间：
2007
期刊：
影响因子：
--
通讯作者：
B. Scholkopf;J. Platt;T. Hofmann
中科院分区：
文献类型：
--
作者：
B. Scholkopf;J. Platt;T. Hofmann
我们正处于多核时代的开端。计算机将拥有越来越多的核心（处理器），但这些架构仍然没有良好的编程框架，因此机器学习没有简单统一的方法来利用潜在的加速。在本文中，我们开发了一个广泛适用的并行编程方法，一个很容易适用于许多不同的学习算法。我们的工作与机器学习中设计（通常是巧妙的）方法来一次加速单个算法的传统形成鲜明对比。具体来说，我们证明了适合统计查询模型[15]的算法可以以某种“求和形式”编写，这使得它们可以在多核计算机上轻松并行化。我们采用Google的map-reduce [7]范式来演示这种并行加速技术在各种学习算法上的应用，包括局部加权线性回归（LWLR），k-means，逻辑回归（LR），朴素贝叶斯（NB），SVM，伊卡，PCA，高斯判别分析（GDA），EM和反向传播（NN）。我们的实验结果表明，基本上线性加速与处理器数量的增加。
We are at the beginning of the multicore era. Computers will have increasingly many cores (processors), but there is still no good programming framework for these architectures, and thus no simple and unified way for machine learning to take advantage of the potential speed up. In this paper, we develop a broadly applicable parallel programming method, one that is easily applied to manydifferent learning algorithms. Our work is in distinct contrast to the tradition in machine learning of designing (often ingenious) ways to speed up a singlealgorithm at a time. Specifically, we show that algorithms that fit the Statistical Query model [15] can be written in a certain “summation form,” which allows them to be easily parallelized on multicore computers. We adapt Google’s map-reduce [7] paradigm to demonstrate this parallel speed up technique on a variety of learning algorithms including locally weighted linear regression (LWLR), k-means, logistic regression (LR), naive Bayes (NB), SVM, ICA, PCA, gaussian discriminant analysis (GDA), EM, and backpropagation (NN). Our experimental results show basically linear speedup with an increasing number of processors.