Federated Learning over Wireless Networks: A Band-limited Coordinated Descent Approach
Federated Learning over Wireless Networks: A Band-limited Coordinated Descent Approach
复制标题
DOI：
10.1109/infocom42981.2021.9488818
复制
发表时间：
2021-02
期刊：
影响因子：
--
通讯作者：
Junshan Zhang;Na Li;M. Dedeoglu
中科院分区：
文献类型：
--
作者：
Junshan Zhang;Na Li;M. Dedeoglu
我们考虑在网络边缘使用多对一无线架构进行联合学习，其中多个边缘设备使用本地数据协作训练模型。无线连接的不可靠性质，加上边缘设备计算资源的限制，决定了边缘设备的本地更新应仔细设计和压缩，以匹配可用的无线通信资源，并应与接收方协同工作。在此基础上，我们提出了基于SGD的带限坐标下降算法。具体地，对于采用空中计算的无线边缘，接收器在每次迭代中选择跨边缘设备的梯度更新的k个坐标的公共子集，然后在k个子载波上同时发送，每个子载波都经历时变的信道条件。我们用所产生的梯度偏差和均方误差来表征通信误差和压缩对所提出算法的收敛的影响。然后，我们通过联合优化功率分配和学习率来研究学习驱动的通信错误最小化。我们的研究结果表明，不同子载波之间的最优功率分配应该同时考虑梯度值和信道条件，从而推广了广泛使用的注水策略。我们还开发了可实施的次优分布式解决方案。
We consider a many-to-one wireless architecture for federated learning at the network edge, where multiple edge devices collaboratively train a model using local data. The unreliable nature of wireless connectivity, together with constraints in computing resources at edge devices, dictates that the local updates at edge devices should be carefully crafted and compressed to match the wireless communication resources available and should work in concert with the receiver. Thus motivated, we propose SGD-based bandlimited coordinate descent algorithms for such settings. Specifically, for the wireless edge employing over-the-air computing, a common subset of k-coordinates of the gradient updates across edge devices are selected by the receiver in each iteration, and then transmitted simultaneously over k sub-carriers, each experiencing time-varying channel conditions. We characterize the impact of communication error and compression, in terms of the resulting gradient bias and mean squared error, on the convergence of the proposed algorithms. We then study learning-driven communication error minimization via joint optimization of power allocation and learning rates. Our findings reveal that optimal power allocation across different sub-carriers should take into account both the gradient values and channel conditions, thus generalizing the widely used water-filling policy. We also develop sub-optimal distributed solutions amenable to implementation.