CoCoA: A General Framework for Communication-Efficient Distributed Optimization
CoCoA: A General Framework for Communication-Efficient Distributed Optimization
复制标题
DOI：
--
复制
发表时间：
2016-11
期刊：
影响因子：
--
通讯作者：
Virginia Smith;Simone Forte;Chenxin Ma;Martin Takác;Michael I. Jordan;Martin Jaggi
中科院分区：
文献类型：
--
作者：
Virginia Smith;Simone Forte;Chenxin Ma;Martin Takác;Michael I. Jordan;Martin Jaggi
现代数据集的规模需要开发用于机器学习的有效分布式优化方法。我们提出了用于分布式计算环境的通用框架可可，该框架具有有效的通信方案，适用于机器学习和信号处理中的各种问题。我们扩展了框架以涵盖一般的非严格键合剂，包括诸如lasso，稀疏的逻辑回归和弹性净正则化的问题，并显示如何将早期的工作作为特殊情况得出。我们为凸的正规损失最小化目标提供了收敛保证，利用了一种新的方法来处理非巧妙的符合正规化器和非平滑损失函数。所得的框架显着提高了对最新方法的性能，正如我们在实际分布式数据集上进行的一系列实验所述所说明的那样。
The scale of modern datasets necessitates the development of efficient distributed optimization methods for machine learning. We present a general-purpose framework for distributed computing environments, CoCoA, that has an efficient communication scheme and is applicable to a wide variety of problems in machine learning and signal processing. We extend the framework to cover general non-strongly-convex regularizers, including L1-regularized problems like lasso, sparse logistic regression, and elastic net regularization, and show how earlier work can be derived as a special case. We provide convergence guarantees for the class of convex regularized loss minimization objectives, leveraging a novel approach in handling non-strongly-convex regularizers and non-smooth loss functions. The resulting framework has markedly improved performance over state-of-the-art methods, as we illustrate with an extensive set of experiments on real distributed datasets.