FLIPS: Federated Learning using Intelligent Participant Selection
FLIPS: Federated Learning using Intelligent Participant Selection
复制标题
DOI：
10.1145/3590140.3629123
复制
发表时间：
2023-08
期刊：
影响因子：
--
通讯作者：
R. Bhope;K. R. Jayaram;N. Venkatasubramanian;Ashish Verma;Gegi Thomas
中科院分区：
文献类型：
--
作者：
R. Bhope;K. R. Jayaram;N. Venkatasubramanian;Ashish Verma;Gegi Thomas
本文介绍了FLIPS的设计和实现，FLIPS是一个中间件系统，用于管理联邦学习（FL）培训工作负载中的数据和参与者异构性。特别是，我们研究了标签分布聚类对联邦学习中参与者选择的好处。FLIPS根据其数据先验的标签分布对参与FL培训工作的各方进行聚类，并在FL培训期间确保每个聚类在选定的参与者中得到公平的代表。FLIPS可以支持最常见的FL算法，包括FedAvg，FedProx，FedDyn，FedOpt和FedYogi。为了管理平台异构性和动态资源可用性，FLIPS采用了一种离散管理机制来处理分布式智能社区应用程序中不断变化的容量。标签分发、聚类和参与者选择的隐私性通过可信执行环境（TEE）来确保。我们的综合实证评估比较了FLIPS与随机参与者选择，以及其他三种“智能”选择机制- Oort [51]，TiFL [15]和梯度聚类[27]，使用四个真实世界的数据集，两种不同的非IID分布和三种常见的FL算法（FedYogi，FedProx和FedAvg）。我们证明了FLIPS显著提高了收敛性，实现了17-20个百分点的更高精度，降低了20-60%的通信成本，并且这些好处在落后参与者的存在下持续存在。
This paper presents the design and implementation of FLIPS, a middleware system to manage data and participant heterogeneity in federated learning (FL) training workloads. In particular, we examine the benefits of label distribution clustering on participant selection in federated learning. FLIPS clusters parties involved in an FL training job based on the label distribution of their data apriori, and during FL training, ensures that each cluster is equitably represented in the participants selected. FLIPS can support the most common FL algorithms, including FedAvg, FedProx, FedDyn, FedOpt and FedYogi. To manage platform heterogeneity and dynamic resource availability, FLIPS incorporates a straggler management mechanism to handle changing capacities in distributed, smart community applications. Privacy of label distributions, clustering and participant selection is ensured through a trusted execution environment (TEE). Our comprehensive empirical evaluation compares FLIPS with random participant selection, as well as three other "smart" selection mechanisms -- Oort [51], TiFL [15] and gradient clustering [27] using four real-world datasets, two different non-IID distributions and three common FL algorithms (FedYogi, FedProx and FedAvg). We demonstrate that FLIPS significantly improves convergence, achieving higher accuracy by 17-20 percentage points with 20-60% lower communication costs, and these benefits endure in the presence of straggler participants.