Echo of Neighbors: Privacy Amplification for Personalized Private Federated Learning with Shuffle Model
Echo of Neighbors: Privacy Amplification for Personalized Private Federated Learning with Shuffle Model
复制标题
DOI：
10.48550/arxiv.2304.05516
复制
发表时间：
2023-02
期刊：
影响因子：
--
通讯作者：
Yi-xiao Liu;Suyun Zhao;Li Xiong;Yuhan Liu;Hong Chen
中科院分区：
文献类型：
--
作者：
Yi-xiao Liu;Suyun Zhao;Li Xiong;Yuhan Liu;Hong Chen
联邦学习作为一种流行的协作训练模式，容易受到隐私攻击。关于用户的态度不同的隐私级别需要在本地得到满足，而全局模型的严格隐私保证也需要集中。个性化局部差异隐私（PLDP）适用于保护用户不同的局部隐私，但仅提供相当于最坏情况下的局部隐私水平的中央隐私保证。因此，实现强大的中央隐私以及个性化的本地隐私与实用程序有前途的模型是一个具有挑战性的问题。在这项工作中，建立了一个通用框架（APES），利用洗牌模型的隐私放大效应，以加强个性化的局部隐私下的模型隐私。为了收紧隐私的约束，我们量化的异构贡献的中央隐私用户的用户。贡献的特点是能够产生“回声”的扰动，每个用户，这是仔细测量所提出的方法邻居发散和剪辑拉普拉斯机制。此外，我们提出了一个改进的框架（S-APES）与后稀疏化技术，以减少在高维场景中的隐私丢失。据我们所知，洗牌对个性化的本地隐私的影响被认为是第一次。我们提供了一个很强的隐私放大效应，并根据现有的方法的一致的局部隐私的基线结果的界限是更紧。实验表明，我们的框架确保了全局模型的可比或更高的准确性。
Federated Learning, as a popular paradigm for collaborative training, is vulnerable against privacy attacks. Different privacy levels regarding users' attitudes need to be satisfied locally, while a strict privacy guarantee for the global model is also required centrally. Personalized Local Differential Privacy (PLDP) is suitable for preserving users' varying local privacy, yet only provides a central privacy guarantee equivalent to the worst-case local privacy level. Thus, achieving strong central privacy as well as personalized local privacy with a utility-promising model is a challenging problem. In this work, a general framework (APES) is built up to strengthen model privacy under personalized local privacy by leveraging the privacy amplification effect of the shuffle model. To tighten the privacy bound, we quantify the heterogeneous contributions to the central privacy user by user. The contributions are characterized by the ability of generating "echos" from the perturbation of each user, which is carefully measured by proposed methods Neighbor Divergence and Clip-Laplace Mechanism. Furthermore, we propose a refined framework (S-APES) with the post-sparsification technique to reduce privacy loss in high-dimension scenarios. To the best of our knowledge, the impact of shuffling on personalized local privacy is considered for the first time. We provide a strong privacy amplification effect, and the bound is tighter than the baseline result based on existing methods for uniform local privacy. Experiments demonstrate that our frameworks ensure comparable or higher accuracy for the global model.