The Poisson Binomial Mechanism for Unbiased Federated Learning with Secure Aggregation
The Poisson Binomial Mechanism for Unbiased Federated Learning with Secure Aggregation
复制标题
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Wei-Ning Chen;Ayfer Özgür;P. Kairouz
中科院分区：
文献类型：
--
作者：
Wei-Ning Chen;Ayfer Özgür;P. Kairouz
我们介绍了泊松二项机制（PBM），一个离散差分隐私机制的分布式均值估计（DME）与联邦学习和分析的应用。我们对其隐私保证进行了严格的分析，表明它实现了与连续高斯机制相同的隐私准确性权衡。我们的分析是基于一个新的约束上的R ′ enyi分歧的两个泊松二项分布，可能是独立的利益。与以前的离散DP计划的基础上加性噪声，我们的机制编码到一个参数的二项分布的本地信息，因此输出分布是离散的有界支持。此外，支持度并不随着隐私预算ε → 0而增加，就像在需要添加更多噪声以实现更高隐私的加法方案的情况下一样;相反，支持度随着ε → 0而变小。有界支持使我们能够联合收割机，我们的机制与安全聚合（SecAgg），多方密码协议，而不需要执行模块裁剪的结果在一个无偏估计的本地向量的总和。这反过来又允许我们将其应用于私人FL设置，并提供SGD算法收敛速度的上限。此外，由于输出分布的支持度随着ε → 0而变小，因此该方案的通信代价随着隐私约束ε的增加而减小，在高隐私或低通信条件下优于所有基于加性噪声的分布式DP方案.
We introduce the Poisson Binomial mechanism (PBM), a discrete differential privacy mechanism for distributed mean estimation (DME) with applications to federated learning and analytics. We provide a tight analysis of its privacy guarantees, showing that it achieves the same privacy-accuracy trade-offs as the continuous Gaussian mechanism. Our analysis is based on a novel bound on the R ´ enyi divergence of two Poisson binomial distributions that may be of independent interest. Unlike previous discrete DP schemes based on additive noise, our mechanism encodes local information into a parameter of the binomial distribution, and hence the output distribution is discrete with bounded support. Moreover, the support does not increase as the privacy budget ε → 0 as in the case of additive schemes which require the addition of more noise to achieve higher privacy; on the contrary, the support becomes smaller as ε → 0 . The bounded support enables us to combine our mechanism with secure aggregation (SecAgg), a multi-party cryptographic protocol, without the need of performing modular clipping which results in an unbiased estimator of the sum of the local vectors. This in turn allows us to apply it in the private FL setting and provide an upper bound on the convergence rate of the SGD algo-rithm. Moreover, since the support of the output distribution becomes smaller as ε → 0 , the communication cost of our scheme decreases with the privacy constraint ε , outperforming all previous distributed DP schemes based on additive noise in the high privacy or low communication regimes.