FedPCC: Parallelism of Communication and Computation for Federated Learning in Wireless Networks
FedPCC: Parallelism of Communication and Computation for Federated Learning in Wireless Networks
复制标题
DOI：
10.1109/tetci.2022.3170471
复制
发表时间：
2022-12
影响因子：
5.3
通讯作者：
Hong Zhang;Hao Tian;M. Dong;K. Ota;Juncheng Jia
中科院分区：
文献类型：
--
作者：
Hong Zhang;Hao Tian;M. Dong;K. Ota;Juncheng Jia
计算技术和通信技术的进步促进了对由移动的设备生成的海量数据的利用。利用这些数据和计算资源来训练高性能机器学习（ML）模型是很有吸引力的。在传统的机器学习方法中，所有数据都上传到服务器进行模型训练，这会导致大量开销和隐私问题。联邦学习（FL）被提出来解决这些问题。在本文中，我们提出了FedPCC，一个有效的方法的基础上的并行通信和计算设备之间的FL在无线网络中。FedPCC考虑了不同设备之间通信和计算能力的差异，并为每一轮中选择的设备优化了训练时间表。FedPCC协议不是将FL训练轮划分为单独的通信和计算步骤，而是安排设备顺序下载全局模型，然后在完成下载后立即开始本地模型训练，这允许更好地利用通信和计算资源。具体来说，我们制定了一个优化问题，以最大限度地减少每个训练轮的时间，并提出了一种算法来解决这个问题的基础上权衡设备的通信和计算能力。FedPCC使用启发式算法，使慢速设备相对更早启动，可以缩短一轮训练的时间，从而减少整个训练时间。我们进行了大量的实验，以证明该协议优于现有的协议在不同的系统设置。
The advances of both computation and communication technologies facilitate the exploitation of massive data generated by mobile devices. It is attractive to leverage these data and computation resources to train high-performance machine learning (ML) models. In traditional ML methods, all data are uploaded to servers for model training, which incur issues of large overhead and privacy concerns. Federated learning (FL) was proposed to address these issues. In this paper, we propose FedPCC, an efficient approach based on the parallelism of communication and computation among devices for FL in wireless networks. FedPCC considers the difference in communication and computation capabilities between different devices and optimizes the training schedule for devices selected in each round. Instead of dividing an FL training round into separate communication and computation steps, FedPCC protocol arranges devices to download the global model sequentially and then start local model training immediately once finishing downloading, which allows better utilization of communication and computation resources. Specifically, we formulate an optimization problem to minimize the time for each training round and propose an algorithm to solve the problem based on the trade-off of devices’ communication and computation capabilities. FedPCC uses a heuristic algorithm to make slow devices start relatively earlier, which can shorten the time in one training round, thus reducing the entire training time. We conduct extensive experiments to demonstrate that the proposed protocol outperforms the existing protocols under different system settings.