Confederated Learning: Federated Learning With Decentralized Edge Servers
Confederated Learning: Federated Learning With Decentralized Edge Servers
复制标题
DOI：
10.1109/tsp.2023.3241768
复制
发表时间：
2022-05
影响因子：
5.4
通讯作者：
Bin Wang;Jun Fang;Hongbin Li;Xiaojun Yuan;Qing Ling
中科院分区：
文献类型：
--
作者：
Bin Wang;Jun Fang;Hongbin Li;Xiaojun Yuan;Qing Ling
联合学习（FL）是一种新兴的机器学习范式，它允许在无需在中央服务器上聚合数据的情况下完成模型训练。大多数关于FL的研究考虑集中式框架，其中单个服务器被赋予中央权限以协调多个设备以迭代方式执行模型训练。由于严格的通信和带宽限制，这种集中式框架随着设备数量的增长而具有有限的可扩展性。为了解决这个问题，在本文中，我们提出了一个联合学习（CFL）框架。拟议的CFL由多个服务器组成，其中每个服务器与传统FL框架中的一组单独的设备连接，并且在服务器之间利用分散的协作，以充分利用分散在整个网络中的数据。提出了一种随机交替方向乘子法（ADMM）算法。所提出的算法采用随机调度策略，随机选择一个子集的设备访问各自的服务器在每次迭代，从而减轻了需要上传大量的信息从设备到服务器。理论分析证明所提出的方法。数值结果表明，该方法可以收敛到一个体面的解决方案显着快于基于梯度的FL算法，从而拥有一个显着的优势，在通信效率方面。
Federated learning (FL) is an emerging machine learning paradigm that allows to accomplish model training without aggregating data at a central server. Most studies on FL consider a centralized framework, in which a single server is endowed with a central authority to coordinate a number of devices to perform model training in an iterative manner. Due to stringent communication and bandwidth constraints, such a centralized framework has limited scalability as the number of devices grows. To address this issue, in this paper, we propose a ConFederated Learning (CFL) framework. The proposed CFL consists of multiple servers, in which each server is connected with an individual set of devices as in the conventional FL framework, and decentralized collaboration is leveraged among servers to make full use of the data dispersed throughout the network. We develop a stochastic alternating direction method of multipliers (ADMM) algorithm for CFL. The proposed algorithm employs a random scheduling policy which randomly selects a subset of devices to access their respective servers at each iteration, thus alleviating the need of uploading a huge amount of information from devices to servers. Theoretical analysis is presented to justify the proposed method. Numerical results show that the proposed method can converge to a decent solution significantly faster than gradient-based FL algorithms, thus boasting a substantial advantage in terms of communication efficiency.