Adaptive Federated Pruning in Hierarchical Wireless Networks
Adaptive Federated Pruning in Hierarchical Wireless Networks
复制标题
DOI：
10.1109/twc.2023.3329450
复制
发表时间：
2023-05
影响因子：
10.4
通讯作者：
Xiaonan Liu;Shiqiang Wang;Yansha Deng;A. Nallanathan
中科院分区：
文献类型：
--
作者：
Xiaonan Liu;Shiqiang Wang;Yansha Deng;A. Nallanathan
联邦学习（FL）是一种很有前途的隐私保护分布式学习框架，其中服务器聚合由多个设备更新的模型，而无需访问其私有数据集。分层FL（HFL）作为设备-边缘-云聚合层次结构，可以享受云服务器对更多数据集的访问以及边缘服务器与设备的高效通信。然而，学习延迟随着HFL网络规模的增加而增加，这是由于边缘服务器和设备的数量不断增加，而本地计算能力和通信带宽有限。为了解决这个问题，在本文中，我们引入模型修剪HFL在无线网络中，以减少神经网络的规模。分析了基于模型剪枝的HFL算法在梯度的l {2}$ -范数上的收敛性，分析了模型剪枝算法的计算和通信延迟，并通过联合优化剪枝率和无线资源分配，给出了在给定延迟阈值下最大化收敛速度的优化问题.通过解耦优化问题，并利用Karush-Kuhn-Tucker（KKT）条件，修剪率和无线资源分配的封闭形式的解决方案。仿真结果表明，我们提出的HFL模型修剪达到类似的学习精度相比，没有模型修剪的HFL，并减少约50%的通信成本。
Federated Learning (FL) is a promising privacy-preserving distributed learning framework where a server aggregates models updated by multiple devices without accessing their private datasets. Hierarchical FL (HFL), as a device-edge-cloud aggregation hierarchy, can enjoy both the cloud server’s access to more datasets and the edge servers’ efficient communications with devices. However, the learning latency increases with the HFL network scale due to the increasing number of edge servers and devices with limited local computation capability and communication bandwidth. To address this issue, in this paper, we introduce model pruning for HFL in wireless networks to reduce the neural network scale. We present the convergence analysis of an upper on the $l_{2}$ -norm of gradients for HFL with model pruning, analyze the computation and communication latency of the proposed model pruning scheme, and formulate an optimization problem to maximize the convergence rate under a given latency threshold by jointly optimizing the pruning ratio and wireless resource allocation. By decoupling the optimization problem and using Karush–Kuhn–Tucker (KKT) conditions, closed-form solutions of pruning ratio and wireless resource allocation are derived. Simulation results show that our proposed HFL with model pruning achieves similar learning accuracy compared with the HFL without model pruning and reduces about 50% communication cost.