HiFlash: Communication-Efficient Hierarchical Federated Learning With Adaptive Staleness Control and Heterogeneity-Aware Client-Edge Association
HiFlash: Communication-Efficient Hierarchical Federated Learning With Adaptive Staleness Control and Heterogeneity-Aware Client-Edge Association
复制标题
DOI：
10.1109/tpds.2023.3238049
复制
发表时间：
2023-05-01
影响因子：
5.3
通讯作者：
Zhang, Junshan
中科院分区：
文献类型：
--
作者：
Wu, Qiong;Chen, Xu;Zhang, Junshan
联邦学习（FL）是一种很有前途的范例，它支持跨大量客户机协作学习共享模型，同时将训练数据保存在本地。然而，对于许多现有的FL系统，客户端需要频繁地直接通过广域网（WAN）与远程云服务器交换大数据规模的模型参数，这导致了巨大的通信开销和较长的传输时间。为了缓解通信瓶颈，我们采用HiFL的分层联邦学习范式，该范式利用移动边缘计算的优势，将同步客户端边缘模型聚合和异步边缘云模型聚合结合在一起，大大减少了广域网传输的流量。具体而言，我们首先从理论上分析了HiFL的收敛界，并确定了模型性能改进的关键可控因素。然后，我们主张通过创新地集成基于深度强化学习的自适应过时控制和异构感知的客户端边缘关联策略来增强HiFlash的设计，以提高系统效率并在不影响模型准确性的情况下减轻过时效应。大量的实验证实了HiFlash在模型精度、减少通信和系统效率方面的优越性能。
Federated learning (FL) is a promising paradigm that enables collaboratively learning a shared model across massive clients while keeping the training data locally. However, for many existing FL systems, clients need to frequently exchange model parameters of large data size with the remote cloud server directly via wide-area networks (WAN), leading to significant communication overhead and long transmission time. To mitigate the communication bottleneck, we resort to the hierarchical federated learning paradigm of HiFL, which reaps the benefits of mobile edge computing and combines synchronous client-edge model aggregation and asynchronous edge-cloud model aggregation together to greatly reduce the traffic volumes of WAN transmissions. Specifically, we first analyze the convergence bound of HiFL theoretically and identify the key controllable factors for model performance improvement. We then advocate an enhanced design of HiFlash by innovatively integrating deep reinforcement learning based adaptive staleness control and heterogeneity-aware client-edge association strategy to boost the system efficiency and mitigate the staleness effect without compromising model accuracy. Extensive experiments corroborate the superior performance of HiFlash in model accuracy, communication reduction, and system efficiency.