Coded Computing for Low-Latency Federated Learning Over Wireless Edge Networks
Coded Computing for Low-Latency Federated Learning Over Wireless Edge Networks
复制标题
DOI：
10.1109/jsac.2020.3036961
复制
发表时间：
2020-11
影响因子：
16.4
通讯作者：
Saurav Prakash;S. Dhakal;M. Akdeniz;Yair Yona;S. Talwar;S. Avestimehr;N. Himayat
中科院分区：
文献类型：
--
作者：
Saurav Prakash;S. Dhakal;M. Akdeniz;Yair Yona;S. Talwar;S. Avestimehr;N. Himayat
联合学习能够从位于客户端节点的数据中训练全局模型，而无需数据共享和将客户端数据移动到集中式服务器。由于客户端之间计算能力和通信链路质量的异构性和随机波动，多访问边缘计算（MEC）网络中的联邦学习性能收敛缓慢。我们提出了一种新的编码计算框架CodedFedL，它将结构化编码冗余注入到联邦学习中，以减轻掉队者并加快训练过程。CodedFedL通过随机傅立叶特征有效地利用分布式内核嵌入，将训练任务转换为计算上有利的分布式线性回归，从而实现非线性联邦学习的编码计算。此外，客户端通过对本地数据集进行编码来生成本地奇偶校验数据集，而服务器将它们组合以获得全局奇偶校验数据集。来自全局奇偶校验数据集的梯度补偿训练期间的离散梯度，从而加快收敛。为了最大限度地减少MEC服务器上的历元截止时间，我们提供了一种易于处理的方法，通过利用计算的统计特性以及通信延迟来找到客户端在训练期间处理的编码冗余量和本地数据点的数量。我们还描述了当客户端与服务器共享本地奇偶校验数据集时数据隐私的泄漏。此外，我们分析了CodedFedL的收敛速度和迭代复杂性简化假设下，通过处理CodedFedL作为一个随机梯度下降算法。最后，为了证明CodedFedL在实践中可以实现的收益，我们使用实际的网络参数和基准数据集进行了数值实验，其中CodedFedL与基准方案相比，整体训练时间加快了15倍。
Federated learning enables training a global model from data located at the client nodes, without data sharing and moving client data to a centralized server. Performance of federated learning in a multi-access edge computing (MEC) network suffers from slow convergence due to heterogeneity and stochastic fluctuations in compute power and communication link qualities across clients. We propose a novel coded computing framework, CodedFedL, that injects structured coding redundancy into federated learning for mitigating stragglers and speeding up the training procedure. CodedFedL enables coded computing for non-linear federated learning by efficiently exploiting distributed kernel embedding via random Fourier features that transforms the training task into computationally favourable distributed linear regression. Furthermore, clients generate local parity datasets by coding over their local datasets, while the server combines them to obtain the global parity dataset. Gradient from the global parity dataset compensates for straggling gradients during training, and thereby speeds up convergence. For minimizing the epoch deadline time at the MEC server, we provide a tractable approach for finding the amount of coding redundancy and the number of local data points that a client processes during training, by exploiting the statistical properties of compute as well as communication delays. We also characterize the leakage in data privacy when clients share their local parity datasets with the server. Additionally, we analyze the convergence rate and iteration complexity of CodedFedL under simplifying assumptions, by treating CodedFedL as a stochastic gradient descent algorithm. Finally, for demonstrating gains that CodedFedL can achieve in practice, we conduct numerical experiments using practical network parameters and benchmark datasets, in which CodedFedL speeds up the overall training time by up to $15\times $ in comparison to the benchmark schemes.