FedSplit: An algorithmic framework for fast federated optimization
FedSplit: An algorithmic framework for fast federated optimization
复制标题
DOI：
--
复制
发表时间：
2020-05
期刊：
影响因子：
--
通讯作者：
Reese Pathak;M. Wainwright
中科院分区：
文献类型：
--
作者：
Reese Pathak;M. Wainwright
受联邦学习的启发，我们考虑了分布式优化的中心辐射模型，在该模型中，一个中央机构在限制通信的同时协调许多代理之间的解决方案的计算。我们首先研究了一些过去的联邦优化程序，并表明，他们的不动点不需要对应于原来的优化问题的固定点，即使在简单的凸设置与确定性更新。为了解决这些问题，我们引入了FedSplit，一类基于算子分裂过程的算法，用于求解具有可加结构的分布式凸极小化问题。我们证明了这些程序有正确的不动点，对应于原优化问题的最优解，我们刻画了它们的收敛速度在不同的设置。我们的理论表明，这些方法是可证明的强大的不精确计算的中间本地量。我们用一些简单的实验来补充我们的理论，这些实验证明了我们的方法在实践中的好处。
Motivated by federated learning, we consider the hub-and-spoke model of distributed optimization in which a central authority coordinates the computation of a solution among many agents while limiting communication. We first study some past procedures for federated optimization, and show that their fixed points need not correspond to stationary points of the original optimization problem, even in simple convex settings with deterministic updates. In order to remedy these issues, we introduce FedSplit, a class of algorithms based on operator splitting procedures for solving distributed convex minimization with additive structure. We prove that these procedures have the correct fixed points, corresponding to optima of the original optimization problem, and we characterize their convergence rates under different settings. Our theory shows that these methods are provably robust to inexact computation of intermediate local quantities. We complement our theory with some simple experiments that demonstrate the benefits of our methods in practice.