Quality-Aware Distributed Computation and Communication Scheduling for Fast Convergent Wireless Federated Learning
Quality-Aware Distributed Computation and Communication Scheduling for Fast Convergent Wireless Federated Learning
复制标题
DOI：
10.23919/wiopt52861.2021.9589802
复制
发表时间：
2021-10
期刊：
影响因子：
--
通讯作者：
Dongsheng Li;Yuxi Zhao;Xiaowen Gong
中科院分区：
文献类型：
--
作者：
Dongsheng Li;Yuxi Zhao;Xiaowen Gong
在无线联邦学习 (WFL) 中，机器学习 (ML) 模型在无线边缘设备上进行分布式训练，无需从设备收集数据。在这种设置中，局部模型更新的质量在很大程度上取决于局部随机梯度的方差，该方差由用于计算更新的小批量数据大小决定。在本文中，我们探索了 WFL 的质量感知分布式计算，其中用户设备共享有限的通信资源，使用小批量大小作为“旋钮”来控制用户本地更新的质量。特别是，我们研究了联合小批量大小设计和通信调度，目标是最小化 FL 算法的训练损失和训练时间。对于 IID 数据的情况，我们首先描述最佳通信调度和最佳小批量大小。然后我们开发了一种贪心算法，以近似比率找到最佳的参与用户集。对于非独立同分布数据的情况，我们首先描述最佳通信结构和最佳小批量大小。然后我们开发算法来找到某些特殊情况的最佳通信顺序。我们的研究结果为 WFL 的计算通信协同设计提供了有用的见解。我们使用模拟评估了所提出的小批量大小设计和通信调度，这证实了学习准确性和学习时间的提高。
In wireless federated learning (WFL), machine learning (ML) models are trained distributively on wireless edge devices without the need of collecting data from the devices. In such a setting, the quality of a local model update heavily depends on the variance of the local stochastic gradient, determined by the mini-batch data size used to compute the update. In this paper, we explore quality-aware distributed computation for WFL where user devices share limited communication resources, using mini-batch size as a "knob" to control the quality of users’ local updates. In particular, we study joint mini-batch size design and communication scheduling, with the goal of minimizing the training loss as well as the training time of the FL algorithm. For the case of IID data, we first characterize the optimal communication scheduling and the optimal minibatch sizes. Then we develop a greedy algorithm that finds the optimal set of participating users with an approximation ratio. For the case of non-IID data, we first characterize the optimal communication structure and the optimal mini-batch sizes. Then we develop algorithms that find the optimal communication order for some special cases. Our findings provide useful insights for the computation-communication co-design for WFL. We evaluate the proposed mini-batch size design and communication scheduling using simulations, which corroborate improved learning accuracy and learning time.