Vector Quantized Compressed Sensing for Communication-Efficient Federated Learning
Vector Quantized Compressed Sensing for Communication-Efficient Federated Learning
复制标题
DOI：
10.1109/gcwkshps56602.2022.10008615
复制
发表时间：
2022-12
期刊：
影响因子：
--
通讯作者：
Yong-Nam Oh;Yo-Seb Jeon;Mingzhe Chen;W. Saad
中科院分区：
文献类型：
--
作者：
Yong-Nam Oh;Yo-Seb Jeon;Mingzhe Chen;W. Saad
在本文中，提出了一种通信效率的联邦学习（FL）框架，它利用矢量量化压缩感知的思想，第一次，在FL中的无线设备的本地模型更新压缩。对于压缩，每个本地模型更新被投影到一个较低的维度空间，然后，投影的本地模型更新量化使用矢量量化器。通过对压缩的局部模型更新的聚合使用稀疏信号恢复算法来重构参数服务器处的全局模型更新。我们的压缩策略的一个关键特征是，投影后的局部模型更新被有效地建模为高斯随机向量的中心极限定理。受此启发，最佳矢量量化器推导出最小化的局部模型更新的压缩误差。MNIST数据集上的仿真结果表明，使用0.5位来表示每个局部模型更新条目的所提出的框架与没有局部更新压缩的FL相比，分类精度降低了不到1%。
In this paper, a communication-efficient federated learning (FL) framework is proposed, which leverages ideas from vector quantized compressed sensing, for the first time, to compress the local model updates at wireless devices in FL. For the compression, each local model update is projected onto a lower dimensional space; then, the projected local model update is quantized by using a vector quantizer. The global model update at a parameter server is reconstructed by using a sparse signal recovery algorithm on the aggregation of the compressed local model updates. A key feature of our compression strategy is that the local model update after the projection is effectively modeled as a Gaussian random vector by the central limit theorem. Inspired by this feature, the optimal vector quantizer is derived for minimizing the compression error of the local model update. Simulation results on the MNIST dataset demonstrate that the proposed framework that uses 0.5 bit to represent each local model update entry shows less than a 1% decrease in classification accuracy compared to FL without local update compression.