On the Tradeoff between Energy, Precision, and Accuracy in Federated Quantized Neural Networks
On the Tradeoff between Energy, Precision, and Accuracy in Federated Quantized Neural Networks
复制标题
DOI：
10.1109/icc45855.2022.9838362
复制
发表时间：
2021-11
期刊：
影响因子：
--
通讯作者：
Minsu Kim;W. Saad;Mohammad Mozaffari;M. Debbah
中科院分区：
文献类型：
--
作者：
Minsu Kim;W. Saad;Mohammad Mozaffari;M. Debbah
在具有资源受限设备的无线网络上部署联邦学习（FL）需要在准确性、能效和精度之间取得平衡。FL上的现有技术通常需要设备使用32位精度水平来训练深度神经网络（DNN）以用于数据表示以提高准确性。然而，这样的算法对于资源受限的设备是不切实际的，因为DNN可能需要执行数百万个操作。因此，具有高精度水平的训练DNN会导致FL的高能量成本。在本文中，提出了一种量化FL框架，该框架表示在本地训练和上行链路传输中具有有限精度水平的数据。在这里，通过使用量化神经网络（QNN）来捕获有限的精度水平，该量化神经网络以固定精度格式对权重和激活进行量化。在所考虑的FL模型中，每个设备训练其QNN并将量化的训练结果发送到基站。能量模型的本地培训和传输的量化严格推导。一个能量最小化的问题制定的精度水平，同时确保收敛。为了解决这个问题，我们首先解析推导FL收敛速度，并使用线搜索方法。仿真结果表明，我们的FL框架可以减少高达53%的能源消耗相比，一个标准的FL模型。研究结果还揭示了在无线网络上FL的精度，能量和准确性之间的权衡。
Deploying federated learning (FL) over wireless networks with resource-constrained devices requires balancing between accuracy, energy efficiency, and precision. Prior art on FL often requires devices to train deep neural networks (DNNs) using a 32-bit precision level for data representation to improve accuracy. However, such algorithms are impractical for resource-constrained devices since DNNs could require execution of millions of operations. Thus, training DNNs with a high precision level incurs a high energy cost for FL. In this paper, a quantized FL framework, that represents data with a finite level of precision in both local training and uplink transmission, is proposed. Here, the finite level of precision is captured through the use of quantized neural networks (QNNs) that quantize weights and activations in fixed-precision format. In the considered FL model, each device trains its QNN and transmits a quantized training result to the base station. Energy models for the local training and the transmission with the quantization are rigorously derived. An energy minimization problem is formulated with respect to the level of precision while ensuring convergence. To solve the problem, we first analytically derive the FL convergence rate and use a line search method. Simulation results show that our FL framework can reduce energy consumption by up to 53% compared to a standard FL model. The results also shed light on the tradeoff between precision, energy, and accuracy in FL over wireless networks.