An Optimal Stopping Approach for Iterative Training in Federated Learning
An Optimal Stopping Approach for Iterative Training in Federated Learning
复制标题
DOI：
10.1109/ciss48834.2020.1570616094
复制
发表时间：
2020-03
期刊：
影响因子：
--
通讯作者：
Pengfei Jiang;Lei Ying
中科院分区：
文献类型：
--
作者：
Pengfei Jiang;Lei Ying
研究了联邦学习中的迭代训练问题。我们考虑具有单个参数服务器（PS）和M个客户端设备的系统，用于利用客户端设备上的分布式数据集来训练预测学习模型。客户端使用公共无线信道与参数服务器通信，因此每次只能有一台设备进行传输。训练是一个由多轮组成的迭代过程。在每一轮（也称为迭代）开始时，每个客户端都用自己的数据训练模型，由参数服务器在该轮开始时广播。在完成训练之后，当无线信道可用时，设备将更新发送到参数服务器。服务器聚合更新以获得新模型，并将其广播给所有客户端以开始新一轮。我们考虑自适应训练，其中参数服务器决定何时停止/重新开始新一轮，并将问题制定为最优停止问题。虽然这个最优停止问题是很难解决的，我们提出了一个修改的最优停止问题。我们首先开发了一个低复杂度的算法来解决修改后的问题，这也适用于原问题。在真实的数据集上的实验表明，与每轮收集固定数量更新的策略相比，该策略有显著的改进。
This paper studies the problem of iterative training in Federated Learning. We consider a system with a single parameter server (PS) and M client devices for training a predictive learning model with distributed data sets on the client devices. The clients communicate with the parameter server using a common wireless channel, so each time only one device can transmit. The training is an iterative process consisting of multiple rounds. At beginning of each round (also called an iteration), each client trains the model, broadcast by the parameter server at the beginning of the round, with its own data. After finishing training, the device transmits the update to the parameter server when the wireless channel is available. The server aggregates updates to obtain a new model and broadcasts it to all clients to start a new round. We consider adaptive training where the parameter server decides when to stop/restart a new round, and formulate the problem as an optimal stopping problem. While this optimal stopping problem is difficult to solve, we propose a modified optimal stopping problem. We first develop a low complexity algorithm to solve the modified problem, which also works for the original problem. Experiments on a real data set shows significant improvements compared with policies collecting a fixed number of updates in each round.