On the Convergence of Multi-Server Federated Learning With Overlapping Area
On the Convergence of Multi-Server Federated Learning With Overlapping Area
复制标题
DOI：
10.1109/tmc.2022.3200016
复制
发表时间：
2023-11-01
影响因子：
7.9
通讯作者：
Liu, Yao
中科院分区：
文献类型：
--
作者：
Qu, Zhe;Li, Xingyu;Liu, Yao
多服务器联合学习被认为是解决单服务器联合学习通信资源有限问题的一种很有前途的解决方案。我们考虑一个典型的多服务器FL架构，其中区域服务器的覆盖区域可能会重叠。该体系结构的关键是位于重叠区域的客户端基于所有可访问区域模型的平均模型来更新其本地模型，从而实现了不同区域服务器之间的间接模型共享。由于复杂的网络拓扑，收敛分析比单服务器FL的收敛分析更具挑战性。在本文中，我们首先针对这种多服务器FL结构提出了一种新的MS-FedAvg算法，并分析了它在一般非凸环境下的非iid数据集上的收敛。由于每个区域服务器上的客户端数量远远少于单服务器FL，因此每个客户端的带宽应该足够大，以便成功地与服务器进行训练模型的通信，这表明在多服务器FL中完全客户端参与是可行的。此外，我们还对部分客户参与方案进行了收敛分析，并提出了一种新的偏向部分参与策略以进一步加速收敛。我们的结果表明，收敛结果高度依赖于所有三种策略中每个区域类型的客户数量与客户总数的比率。大量的实验结果表明，该算法具有很好的性能，支持了我们的理论结果。
Multi-server Federated learning (FL) has been considered as a promising solution to address the limited communication resource problem of single-server FL. We consider a typical multi-server FL architecture, where the coverage areas of regional servers may overlap. The key point of this architecture is that the clients located in the overlapping areas update their local models based on the average model of all accessible regional models, which enables indirect model sharing among different regional servers. Due to the complicated network topology, the convergence analysis is much more challenging than in single-server FL. In this paper, we firstly propose a novel MS-FedAvg algorithm for this multi-server FL architecture and analyze its convergence on non-iid datasets for general non-convex settings. Since the number of clients located in each regional server is much less than single-server FL, the bandwidth of each client should be large enough to successfully communicate training models with the server, which indicates that full client participation can work in multi-server FL. Also, we provide the convergence analysis of the partial client participation scheme and develop a new biased partial participation strategy to further accelerate convergence. Our results indicate that the convergence results highly depend on the ratio of the number of clients in each area type to the total number of clients in all three strategies. The extensive experiments show remarkable performance and support our theoretical results.