PyramidFL: a fine-grained client selection framework for efficient federated learning
PyramidFL: a fine-grained client selection framework for efficient federated learning
复制标题
DOI：
10.1145/3495243.3517017
复制
发表时间：
2022-10
期刊：
影响因子：
--
通讯作者：
Chenning Li;Xiao Zeng;Mi Zhang;Zhichao Cao
中科院分区：
文献类型：
--
作者：
Chenning Li;Xiao Zeng;Mi Zhang;Zhichao Cao
联邦学习 (FL) 是一种新兴的分布式机器学习 (ML) 范式，具有增强的隐私性，旨在为尽可能多的参与者实现“良好”的 ML 模型，同时消耗尽可能少的时间。通过在数千甚至数百万客户中执行，FL 在参与者之间表现出异构的统计特征和系统差异，这使得其在采用传统 ML 范式时的训练受到影响。训练效率下降的根本原因是随机的客户端选择标准。尽管现有的 FL 范例提出了几种用于客户端选择的优化方案，但由于对客户端数据和系统异构性的开发不足，它们仍然是粗粒度的，从而为各种 FL 应用程序产生了次优性能。在本文中，我们提出 PyramidFL1 来加速 FL 训练，同时实现更高的最终模型性能（即准确时间）。 PyramidFL的核心是细粒度的客户选择，其中PyramidFL不仅关注选定参与者和未选定参与者的差异进行客户选择，而且充分利用选定客户内的数据和系统异构性，更有效地分析其效用。具体来说，PyramidFL 首先从全局（即服务器）视图中确定基于实用程序的客户端选择，然后在本地（即客户端）优化其实用程序分析以进行进一步的客户端选择。通过这种方式，我们可以一致地优先使用那些具有较高统计和系统利用率的客户端。与最先进的（即 Oort）相比，我们对开源 FL 基准的评估表明，PyramidFL 将最终模型精度提高了 3.68% - 7.33%，挂钟时间消耗加速了 2.71 x - 13.66 倍。
Federated learning (FL) is an emerging distributed machine learning (ML) paradigm with enhanced privacy, aiming to achieve a "good" ML model for as many as participants while consuming as little as wall clock time. By executing across thousands or even millions of clients, FL demonstrates heterogeneous statistical characteristics and system divergence widely across participants, making its training suffer when adopting the traditional ML paradigm. The root cause of the training efficiency degradation is the random client selection criteria. Although existing FL paradigms propose several optimization schemes for client selection, they are still coarse-grained due to their under-exploitation on the clients' data and system heterogeneity, yielding sub-optimal performance for a variety of FL applications. In this paper, we propose PyramidFL1 to speed up the FL training while achieving a higher final model performance (i.e., time-to-accuracy). The core of PyramidFL is a fine-grained client selection, in which PyramidFL does not only focus on the divergence of those selected participants and non-selected ones for client selection but also fully exploits the data and system heterogeneity within selected clients to profile their utility more efficiently. Specifically, PyramidFL first determines the utility-based client selection from the global (i.e., server) view and then optimizes its utility profiling locally (i.e., client) for further client selection. In this way, we can prioritize the use of those clients with higher statistical and system utility consistently. In comparison with the state-of-the-art (i.e., Oort), our evaluation on the open-source FL benchmark shows that PyramidFL improves the final model accuracy by 3.68% -- 7.33%, with a speedup of 2.71 x -- 13.66X on the wall clock time consumption.