L-DQN: An Asynchronous Limited-Memory Distributed Quasi-Newton Method
L-DQN: An Asynchronous Limited-Memory Distributed Quasi-Newton Method
复制标题
DOI：
10.1109/cdc45484.2021.9682985
复制
发表时间：
2021-08
期刊：
影响因子：
--
通讯作者：
Bugra Can;Saeed Soori;M. Dehnavi;M. Gürbüzbalaban
中科院分区：
文献类型：
--
作者：
Bugra Can;Saeed Soori;M. Dehnavi;M. Gürbüzbalaban
这项工作提出了一种在主/工人通信模型下解决经验风险最小化问题的分布式算法，称为 L-DQN。 L-DQN 是一种分布式有限内存拟牛顿方法，支持工作节点之间的异步计算。我们的方法在存储和通信成本方面都很高效，即在每次迭代中，主节点和工作节点通信大小为 O(d) 的向量，其中 d 是决策变量的维度，每个节点所需的内存量为 O(md)，其中 m 是可调整参数。据我们所知，这是第一个分布式拟牛顿方法，在节点之间存在延迟的异步设置中具有可证明的全局线性收敛保证。提供数值实验来说明我们方法的理论和实际性能。
This work proposes a distributed algorithm for solving empirical risk minimization problems, called L-DQN, under the master/worker communication model. L-DQN is a distributed limited-memory quasi-Newton method that supports asynchronous computations among the worker nodes. Our method is efficient both in terms of storage and communication costs, i.e., in every iteration, the master node and workers communicate vectors of size O(d), where d is the dimension of the decision variable, and the amount of memory required on each node is O(md), where m is an adjustable parameter. To our knowledge, this is the first distributed quasi-Newton method with provable global linear convergence guarantees in the asynchronous setting where delays between nodes are present. Numerical experiments are provided to illustrate the theory and the practical performance of our method.