Network Update Compression for Federated Learning
Network Update Compression for Federated Learning
复制标题
DOI：
10.1109/vcip49819.2020.9301815
复制
发表时间：
2020-12
期刊：
影响因子：
--
通讯作者：
B. Kathariya;Li Li-Li;Zhu Li;Ling-yu Duan;Shan Liu
中科院分区：
文献类型：
--
作者：
B. Kathariya;Li Li-Li;Zhu Li;Ling-yu Duan;Shan Liu
在联合学习环境中，使用本地生成的数据在各种边缘设备上训练模型，每轮只将当前模型中的更新而不是模型本身发送到服务器，在那里它们被聚合以构成改进的模型。然而，这些边缘设备驻留在具有较高延迟和较低吞吐量连接的高度不均匀的网络中，并且间歇性地可用于培训。此外，网络连接具有下行链路和上行链路的非对称性质。在这项工作中，我们提出了一种高效的编码S解决方案，通过推导更新所需的参数总数来显著减少上行通信开销。这是通过应用高斯混合模型(GMM)在模型间子空间上对Karhuen-Loève变换(KLT)进行局部化并用两个低秩矩阵来表示来实现的。在卷积神经网络(CNN)模型上的实验表明，该模型在保持合理精度的前提下，显著降低了联邦学习中的上行通信开销。
In federated learning setting, models are trained in a variety of edge-devices with locally generated data and each round only updates in the current model rather than the model itself are sent to the server where they are aggregated to compose an improved model. These edge devices, however, reside in highly uneven nature of network with higher latency and lower-throughput connections and are intermittently available for training. In addition, a network connection has an asymmetric nature of downlink and uplink. All these contribute to a major challenge while synchronizing these updates to the server.In this work, we proposed an efficient c oding s olution to significantly r educe u plink c ommunication c ost b y r educing the total number of parameters required for updates. This was achieved by applying Gaussian Mixture Model (GMM) to localize Karhunen–Loève Transform (KLT) on inter-model subspace and representing it with two low-rank matrices. Experiments on convolutional neural network (CNN) models showed the proposed model can significantly reduce the uplink communication cost in federated learning while preserving reasonable accuracy.