Distributed Gradient Methods for Convex Machine Learning Problems in Networks: Distributed Optimization
Distributed Gradient Methods for Convex Machine Learning Problems in Networks: Distributed Optimization
复制标题
DOI：
10.1109/msp.2020.2975210
复制
发表时间：
2020-05
影响因子：
14.9
通讯作者：
A. Nedić
中科院分区：
文献类型：
--
作者：
A. Nedić
本文概述了分布式梯度方法在由m个嵌入在通信网络中的智能体组成的系统中解决形式为minx∈Rn (1/m)∑i = 1m fi(x)的凸机器学习问题。每个智能体i都有一个由其私有目标函数fi(x)捕获的数据集合。这里考虑的分布式算法遵循两个简单的规则：私有的代理函数fi(x)不能向网络中的任何其他代理公开，每个代理都知道网络的本地连接结构，即它只知道它的一跳邻居。agent执行的分布式算法在遵守这两条规则的同时，应该在有限的目标函数知识和有限的局部通信条件下，找到解决整个系统问题的方法。本文概述了此类算法，这些算法通常涉及两个更新步骤：基于代理局部目标函数的梯度步骤和混合步骤，混合步骤本质上是将相关信息从一个代理扩散到网络中的所有其他代理。
This article provides an overview of distributed gradient methods for solving convex machine learning problems of the form minx ∈ Rn (1/m) ∑i = 1m fi(x) in a system consisting of m agents that are embedded in a communication network. Each agent i has a collection of data captured by its privately known objective function fi(x). The distributed algorithms considered here obey two simple rules: privately known agent functions fi(x) cannot be disclosed to any other agent in the network and every agent is aware of the local connectivity structure of the network, i.e., it knows its one-hop neighbors only. While obeying these two rules, the distributed algorithms that agents execute should find a solution to the overall system problem with the limited knowledge of the objective function and limited local communications. Given in this article is an overview of such algorithms that typically involve two update steps: a gradient step based on the agent local objective function and a mixing step that essentially diffuses relevant information from one to all other agents in the network.