On the Stability Analysis of Open Federated Learning Systems
On the Stability Analysis of Open Federated Learning Systems
复制标题
DOI：
10.23919/acc55779.2023.10156023
复制
发表时间：
2022-09
期刊：
影响因子：
--
通讯作者：
Youbang Sun;H. Fernando;Tianyi Chen;Shahin Shahrampour
中科院分区：
文献类型：
--
作者：
Youbang Sun;H. Fernando;Tianyi Chen;Shahin Shahrampour
我们考虑开放式联邦学习 (FL) 系统，客户可以在 FL 过程中加入和/或离开系统。鉴于当前客户端数量的可变性，在开放系统中无法保证收敛到固定模型。相反，我们采用一种新的性能指标，称为开放 FL 系统的稳定性，它量化了开放系统中学习模型的大小。假设局部客户函数是强凸且平滑的，我们从理论上量化了两种 FL 算法的稳定性半径，即局部 SGD 和局部 Adam。我们观察到该半径依赖于几个关键参数，包括函数条件数以及随机梯度的方差。我们的理论结果通过合成数据的数值模拟得到进一步验证。
We consider the open federated learning (FL) systems, where clients may join and/or leave the system during the FL process. Given the variability of the number of present clients, convergence to a fixed model cannot be guaranteed in open systems. Instead, we resort to a new performance metric that we term the stability of open FL systems, which quantifies the magnitude of the learned model in open systems. Under the assumption that local clients’ functions are strongly convex and smooth, we theoretically quantify the radius of stability for two FL algorithms, namely local SGD and local Adam. We observe that this radius relies on several key parameters, including the function condition number as well as the variance of the stochastic gradient. Our theoretical results are further verified by numerical simulations on synthetic data.