Energy Efficient Federated Learning Over Heterogeneous Mobile Devices via Joint Design of Weight Quantization and Wireless Transmission
Energy Efficient Federated Learning Over Heterogeneous Mobile Devices via Joint Design of Weight Quantization and Wireless Transmission
复制标题
DOI：
10.1109/tmc.2022.3213766
复制
发表时间：
2020-12
影响因子：
7.9
通讯作者：
Rui Chen;Liang Li;Kaiping Xue;Chi Zhang;Miao Pan;Yuguang Fang
中科院分区：
文献类型：
--
作者：
Rui Chen;Liang Li;Kaiping Xue;Chi Zhang;Miao Pan;Yuguang Fang
联邦学习（FL）是一种流行的跨移动设备的协作分布式机器学习范例。然而，在资源受限的移动设备上，实际的FL面临着多重挑战，例如，FL中的本地设备上训练和模型更新对移动设备来说是耗电和无线电资源密集的。为了解决这些挑战，在本文中，我们试图将FL纳入未来无线网络的设计中，并为移动设备上的节能FL开发一种新的无线传输和权重量化联合设计。具体来说，我们开发了灵活的权重量化方案，以促进异构移动设备上的设备本地训练。基于局部计算能耗与模型更新能耗相当的观察，我们将节能FL问题转化为混合整数规划问题，在保证模型性能和训练延迟的同时，共同确定异构移动设备的量化和频谱资源分配策略，以最小化整体FL能耗（计算+传输）。由于该问题的优化变量是强耦合的，提出了一种高效的迭代算法，推导了带宽分配和权重量化水平。通过大量的仿真验证了所提方案的有效性。
Federated learning (FL) is a popular collaborative distributed machine learning paradigm across mobile devices. However, practical FL over resource constrained mobile devices confronts multiple challenges, e.g., the local on-device training and model updates in FL are power hungry and radio resource intensive for mobile devices. To address these challenges, in this paper, we attempt to take FL into the design of future wireless networks and develop a novel joint design of wireless transmission and weight quantization for energy efficient FL over mobile devices. Specifically, we develop flexible weight quantization schemes to facilitate on-device local training over heterogeneous mobile devices. Based on the observation that the energy consumption of local computing is comparable to that of model updates, we formulate the energy efficient FL problem into a mixed-integer programming problem where the quantization and spectrum resource allocation strategies are jointly determined for heterogeneous mobile devices to minimize the overall FL energy consumption (computation + transmissions) while guaranteeing model performance and training latency. Since the optimization variables of the problem are strongly coupled, an efficient iterative algorithm is proposed, where the bandwidth allocation and weight quantization levels are derived. Extensive simulations are conducted to verify the effectiveness of the proposed scheme.