Federated Learning with Sparsification-Amplified Privacy and Adaptive Optimization
Federated Learning with Sparsification-Amplified Privacy and Adaptive Optimization
复制标题
DOI：
10.24963/ijcai.2021/202
复制
发表时间：
2020-08
期刊：
影响因子：
--
通讯作者：
Rui Hu;Yanmin Gong;Yuanxiong Guo
中科院分区：
文献类型：
--
作者：
Rui Hu;Yanmin Gong;Yuanxiong Guo
联邦学习（FL）使分布式代理能够协作学习集中式模型，而无需彼此共享原始数据。然而，数据局部性不提供足够的隐私保护，并且期望以严格的差分隐私（DP）保证来促进FL。现有的DP机制会引入与模型大小成比例的随机噪声，这在深度神经网络中可能非常大。在本文中，我们提出了一个新的FL框架与稀疏放大隐私。我们的方法集成了随机稀疏与梯度扰动每个代理放大隐私保证。由于稀疏化会增加达到一定目标精度所需的通信轮数，这对DP保证不利，因此我们进一步引入加速技术以帮助降低隐私成本。我们严格分析了我们方法的收敛性，并利用Renyi DP来严格考虑端到端的DP保证。在基准数据集上的大量实验验证了该方法在隐私保证和通信效率方面都优于以前的差分私有FL方法。
Federated learning (FL) enables distributed agents to collaboratively learn a centralized model without sharing their raw data with each other. However, data locality does not provide sufficient privacy protection, and it is desirable to facilitate FL with rigorous differential privacy (DP) guarantee. Existing DP mechanisms would introduce random noise with magnitude proportional to the model size, which can be quite large in deep neural networks. In this paper, we propose a new FL framework with sparsification-amplified privacy. Our approach integrates random sparsification with gradient perturbation on each agent to amplify privacy guarantee. Since sparsification would increase the number of communication rounds required to achieve a certain target accuracy, which is unfavorable for DP guarantee, we further introduce acceleration techniques to help reduce the privacy cost. We rigorously analyze the convergence of our approach and utilize Renyi DP to tightly account the end-to-end DP guarantee. Extensive experiments on benchmark datasets validate that our approach outperforms previous differentially-private FL approaches in both privacy guarantee and communication efficiency.