A One-Sample Decentralized Proximal Algorithm for Non-Convex Stochastic Composite Optimization
A One-Sample Decentralized Proximal Algorithm for Non-Convex Stochastic Composite Optimization
复制标题
DOI：
10.48550/arxiv.2302.09766
复制
发表时间：
2023-02
期刊：
影响因子：
--
通讯作者：
Tesi Xiao;Xuxing Chen;K. Balasubramanian;Saeed Ghadimi
中科院分区：
文献类型：
--
作者：
Tesi Xiao;Xuxing Chen;K. Balasubramanian;Saeed Ghadimi
我们专注于分散随机非凸优化，其中$n$代理共同努力，以优化复合目标函数，这是一个光滑项和非光滑凸项的总和。为了解决这个问题，我们提出了两个单时间尺度算法：Prox-DASA和Prox-DASA-GT。这些算法可以在$\mathcal{O}（n^{-1}\mathcal ^{-2}）$次迭代中找到$\mathcal $-平稳点（即，$\mathcal{O}（1）$）。与以前的工作不同，我们的算法实现了相当的复杂性，而不需要大批量，更复杂的每次迭代操作（如双循环），或更强的假设。我们的理论研究结果得到了广泛的数值实验的支持，这表明我们的算法优于以前的方法。我们的代码可在https://github.com/xuxingc/ProxDASA上获得。
We focus on decentralized stochastic non-convex optimization, where $n$ agents work together to optimize a composite objective function which is a sum of a smooth term and a non-smooth convex term. To solve this problem, we propose two single-time scale algorithms: Prox-DASA and Prox-DASA-GT. These algorithms can find $\epsilon$-stationary points in $\mathcal{O}(n^{-1}\epsilon^{-2})$ iterations using constant batch sizes (i.e., $\mathcal{O}(1)$). Unlike prior work, our algorithms achieve comparable complexity without requiring large batch sizes, more complex per-iteration operations (such as double loops), or stronger assumptions. Our theoretical findings are supported by extensive numerical experiments, which demonstrate the superiority of our algorithms over previous approaches. Our code is available at https://github.com/xuxingc/ProxDASA.