A Decentralized Primal-Dual Framework for Non-Convex Smooth Consensus Optimization
A Decentralized Primal-Dual Framework for Non-Convex Smooth Consensus Optimization
复制标题
DOI：
10.1109/tsp.2023.3239799
复制
发表时间：
2021-07
影响因子：
5.4
通讯作者：
Gabriel Mancino-Ball;Yangyang Xu;Jiewei Chen
中科院分区：
文献类型：
--
作者：
Gabriel Mancino-Ball;Yangyang Xu;Jiewei Chen
在这项工作中，我们介绍ADAPD，一个分散的原始-对偶算法框架，用于解决分布式代理网络上的非凸和光滑的共识优化问题。拟议的框架依赖于一种新型的问题公式，该公式会引发ADMM类型的更新，其中每个代理首先使用其选择的任何方法不精确地解决局部强凸子问题，然后执行邻居通信以更新一组对偶变量。我们提出了两种变体，允许一个单一的梯度步骤的原始更新或多个通信的双重更新，利用每次迭代的成本和迭代次数之间的权衡。当多个通信被执行时，ADAPD可以实现理论上最优的通信复杂度结果的非凸和光滑的共识问题。在几个应用程序（包括深度学习应用程序）上进行的数值实验证明了ADAPD优于几种常用的分散式方法。
In this work, we introduce ADAPD, A DecentrAlized Primal-Dual algorithmic framework for solving non-convex and smooth consensus optimization problems over a network of distributed agents. The proposed framework relies on a novel problem formulation that elicits ADMM-type updates, where each agent first inexactly solves a local strongly convex subproblem with any method of its choice and then performs a neighbor communication to update a set of dual variables. We present two variants that allow for a single gradient step for the primal updates or multiple communications for the dual updates, to exploit the tradeoff between the per-iteration cost and the number of iterations. When multiple communications are performed, ADAPD can achieve theoretically optimal communication complexity results for non-convex and smooth consensus problems. Numerical experiments on several applications, including a deep-learning one, demonstrate the superiority of ADAPD over several popularly used decentralized methods.