Privacy Amplification for Federated Learning via User Sampling and Wireless Aggregation
Privacy Amplification for Federated Learning via User Sampling and Wireless Aggregation
复制标题
DOI：
10.1109/jsac.2021.3118408
复制
发表时间：
2021-12
影响因子：
16.4
通讯作者：
Mohamed Seif Eldin Mohamed-Mohamed-Seif-Eldin-Mohamed-2151239787;Wei-Ting Chang;Ravi Tandon
中科院分区：
文献类型：
--
作者：
Mohamed Seif Eldin Mohamed-Mohamed-Seif-Eldin-Mohamed-2151239787;Wei-Ting Chang;Ravi Tandon
在本文中，我们研究了在中心和局部差分隐私（DP/LDP）约束下，由衰落多址信道建模的具有用户采样的无线信道上的联邦学习问题。研究表明，无线信道的叠加特性提供了带宽高效梯度聚合的双重好处，同时也为用户提供了强大的DP保证。具体来说，中心DP隐私泄露已经被证明是按$\mathcal {O}(1/K^{1/2})$的比例计算的，其中$K$是用户的数量。研究还表明，用户采样与正交传输相结合可以在相同的尺度行为下增强中心DP隐私泄漏。在这项工作中，我们表明，通过联合结合无线聚合和用户抽样，可以获得更强的隐私保证。我们提出了一种私有无线梯度聚合方案，该方案依赖于每个用户独立的随机参与决策。我们提出的方案的中心DP泄漏尺度为$\mathcal {O}(1/K^{3/4})$。此外，我们还发现LDP也受到用户抽样的提升。我们还对所提出方案的收敛速度进行了分析，并在参数服务器上采样的参与者数量为$(a)$已知或$(b)$未知的两种情况下，从理论上和经验上研究了无线资源、收敛和隐私之间的权衡。
In this paper, we study the problem of federated learning over a wireless channel with user sampling, modeled by a fading multiple access channel, subject to central and local differential privacy (DP/LDP) constraints. It has been shown that the superposition nature of the wireless channel provides a dual benefit of bandwidth efficient gradient aggregation, in conjunction with strong DP guarantees for the users. Specifically, the central DP privacy leakage has been shown to scale as $\mathcal {O}(1/K^{1/2})$ , where $K$ is the number of users. It has also been shown that user sampling coupled with orthogonal transmission can enhance the central DP privacy leakage with the same scaling behavior. In this work, we show that, by jointly incorporating both wireless aggregation and user sampling, one can obtain even stronger privacy guarantees. We propose a private wireless gradient aggregation scheme, which relies on independently randomized participation decisions by each user. The central DP leakage of our proposed scheme scales as $\mathcal {O}(1/K^{3/4})$ . In addition, we show that LDP is also boosted by user sampling. We also present analysis for the convergence rate of the proposed scheme and study the tradeoffs between wireless resources, convergence, and privacy theoretically and empirically for two scenarios when the number of sampled participants are $(a)$ known, or $(b)$ unknown at the parameter server.