Towards Understanding Biased Client Selection in Federated Learning
Towards Understanding Biased Client Selection in Federated Learning
复制标题
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Yae Jee Cho;Jianyu Wang;Gauri Joshi
中科院分区：
文献类型：
--
作者：
Yae Jee Cho;Jianyu Wang;Gauri Joshi
联邦学习是一种分布式优化范式，它使大量资源有限的客户端节点能够在不共享数据的情况下协作训练模型。以前的工作分析了联邦学习的融合，会计数据异构性，通信/计算的限制，和部分客户端的参与。然而，大多数假设无偏的客户端参与，其中客户端被选择为使得聚合模型更新是无偏的。在我们的工作中，我们提出了有偏见的客户端选择的联邦学习的收敛性分析，并量化了偏见如何影响收敛速度。我们发现，偏向客户端的选择对客户端具有较高的本地损失产生更快的误差收敛。从这一点来看，我们提出了选择的力量，一个通信和计算效率的客户端选择框架，它灵活地跨越了收敛速度和解决方案偏差之间的权衡。大量的实验表明，Power-of-Choice比基线随机选择的收敛速度快3倍，测试精度高10%。
Federated learning is a distributed optimization paradigm that enables a large number of resource-limited client nodes to cooperatively train a model without data sharing. Previous works analyzed the convergence of federated learning by accounting of data heterogeneity, communication/computation limitations, and partial client participation. However, most assume unbiased client participation, where clients are selected such that the aggregated model update is unbiased. In our work, we present the convergence analysis of federated learning with biased client selection and quantify how the bias affects convergence speed. We show that biasing client selection towards clients with higher local loss yields faster error convergence. From this insight, we propose Power-of-Choice , a communication-and computation-efficient client selection framework that flexibly spans the trade-off between convergence speed and solution bias. Extensive experiments demonstrate that Power-of-Choice can converge up to 3 × faster and give 10% higher test accuracy than the baseline random selection.