Green, Quantized Federated Learning Over Wireless Networks: An Energy-Efficient Design
Green, Quantized Federated Learning Over Wireless Networks: An Energy-Efficient Design
复制标题
DOI：
10.1109/twc.2023.3289177
复制
发表时间：
2022-07
影响因子：
10.4
通讯作者：
Minsu Kim;W. Saad;Mohammad Mozaffari;M. Debbah
中科院分区：
文献类型：
--
作者：
Minsu Kim;W. Saad;Mohammad Mozaffari;M. Debbah
由于设备可用资源有限，在无线网络上实际部署联邦学习（FL）需要平衡能源效率、收敛速度和目标准确性。FL的现有技术通常使用32位精度级别训练深度神经网络（dnn）以实现高精度和快速收敛。然而，这样的场景对于资源受限的设备来说是不切实际的，因为dnn通常具有很高的计算复杂性和内存要求。因此，有必要降低深度神经网络的精度水平，以减少能量消耗。本文提出了一种绿色量化FL框架，该框架既表示局部训练数据，又表示上行传输数据的有限精度级别。在这里，通过使用量化神经网络（QNNs）捕获有限精度水平，该网络以固定精度格式量化权重和激活。在考虑的FL模型中，每个设备训练自己的QNN，并将量化的训练结果传输给基站。严格推导了局部训练和量化传输的能量模型。为了使能量消耗和通信轮数同时最小化，在目标精度约束下，考虑局部迭代次数、选择的设备数量、局部训练和传输的精度等级，建立了多目标优化问题。为了解决这一问题，本文对系统控制变量的收敛速度进行了解析推导。然后，对问题的Pareto边界进行了表征，利用常规边界检查方法给出了有效的解。在实现目标精度的同时平衡两个目标之间的权衡的设计见解是通过使用纳什讨价还价解决方案和分析派生的收敛率得出的。仿真结果表明，与完全精确表示数据的基准FL算法相比，所提出的FL框架可以在不损害收敛速度的情况下将能耗降低高达70%。
The practical deployment of federated learning (FL) over wireless networks requires balancing energy efficiency, convergence rate, and a target accuracy due to the limited available resources of devices. Prior art on FL often trains deep neural networks (DNNs) to achieve high accuracy and fast convergence using 32 bits of precision level. However, such scenarios will be impractical for resource-constrained devices since DNNs typically have high computational complexity and memory requirements. Thus, there is a need to reduce the precision level in DNNs to reduce the energy expenditure. In this paper, a green-quantized FL framework, which represents data with a finite precision level in both local training and uplink transmission, is proposed. Here, the finite precision level is captured through the use of quantized neural networks (QNNs) that quantize weights and activations in fixed-precision format. In the considered FL model, each device trains its QNN and transmits a quantized training result to the base station. Energy models for the local training and the transmission with quantization are rigorously derived. To minimize the energy consumption and the number of communication rounds simultaneously, a multi-objective optimization problem is formulated with respect to the number of local iterations, the number of selected devices, and the precision levels for both local training and transmission while ensuring convergence under a target accuracy constraint. To solve this problem, the convergence rate of the proposed FL system is analytically derived with respect to the system control variables. Then, the Pareto boundary of the problem is characterized to provide efficient solutions using the normal boundary inspection method. Design insights on balancing the tradeoff between the two objectives while achieving a target accuracy are drawn from using the Nash bargaining solution and analyzing the derived convergence rate. Simulation results show that the proposed FL framework can reduce energy consumption until convergence by up to 70% compared to a baseline FL algorithm that represents data with full precision without damaging the convergence rate.