Optimizing the Collaboration Structure in Cross-Silo Federated Learning
Optimizing the Collaboration Structure in Cross-Silo Federated Learning
复制标题
DOI：
10.48550/arxiv.2306.06508
复制
发表时间：
2023-06
期刊：
影响因子：
--
通讯作者：
Wenxuan Bao;Haohan Wang;Jun Wu;Jingrui He
中科院分区：
文献类型：
--
作者：
Wenxuan Bao;Haohan Wang;Jun Wu;Jingrui He
在联邦学习（FL）中，多个客户端协同一起训练机器学习模型，同时保持其数据的分散性。通过利用更多的训练数据，联邦学习面临潜在的负迁移问题：全局联邦学习模型甚至可能比仅使用本地数据训练的模型表现更差。在本文中，我们提出了FedCollab，这是一种新颖的联邦学习框架，它通过根据客户端的分布距离和数据量将其聚类为不重叠的联盟来缓解负迁移。因此，每个客户端仅与具有相似数据分布的客户端协作，并且当它的数据较少时倾向于与更多的客户端协作。我们使用各种数据集、模型和非独立同分布（non - IIDness）类型对我们的框架进行评估。我们的结果表明，FedCollab有效地缓解了多种联邦学习算法中的负迁移，并且始终优于其他聚类联邦学习算法。
In federated learning (FL), multiple clients collaborate to train machine learning models together while keeping their data decentralized. Through utilizing more training data, FL suffers from the potential negative transfer problem: the global FL model may even perform worse than the models trained with local data only. In this paper, we propose FedCollab, a novel FL framework that alleviates negative transfer by clustering clients into non-overlapping coalitions based on their distribution distances and data quantities. As a result, each client only collaborates with the clients having similar data distributions, and tends to collaborate with more clients when it has less data. We evaluate our framework with a variety of datasets, models, and types of non-IIDness. Our results demonstrate that FedCollab effectively mitigates negative transfer across a wide range of FL algorithms and consistently outperforms other clustered FL algorithms.