Agent-Level Differentially Private Federated Learning via Compressed Model Perturbation
Agent-Level Differentially Private Federated Learning via Compressed Model Perturbation
复制标题
DOI：
10.1109/cns56114.2022.9947266
复制
发表时间：
2022-10
期刊：
影响因子：
--
通讯作者：
Yuanxiong Guo;Rui Hu;Yanmin Gong
中科院分区：
文献类型：
--
作者：
Yuanxiong Guo;Rui Hu;Yanmin Gong
联邦学习（FL）涉及分布式代理的网络，这些代理协作学习公共模型，而无需共享原始数据。隐私和沟通是外语的两个重要关注点，但在文献中它们往往被分开处理。虽然可以在FL过程中添加随机噪声以抵御隐私推断攻击，但其大小与模型大小成线性比例，这对于现代深度神经网络来说可能非常大，并导致模型精度严重下降。另一方面，已经提出了各种压缩技术来提高联邦学习的通信效率，但它们与隐私保护的相互作用在很大程度上被忽视了。受隐私保护和通信减少密切相关的背景下，FL的观察，我们提出了一个新的联邦学习计划，称为CMP美联储，实现代理级差分隐私与高模型精度利用通信压缩技术在FL与大模型大小。CMP-Fed的关键组件是压缩模型扰动（CMP），它首先压缩共享模型更新，然后在联邦学习的每一轮通信中使用随机噪声对其进行扰动。基于Fashion-MNIST数据集的实验结果表明，CMP-Fed在相同的隐私保证下，在模型准确性方面大大优于现有的差分私有联邦学习方案，同时仍然享有模型压缩的通信优势。
Federated learning (FL) involves a network of distributed agents that collaboratively learn a common model without sharing their raw data. Privacy and communication are two critical concerns of FL, but they are often treated separately in the literature. While random noise can be added during the FL process to defend against privacy inference attacks, its magnitude is linearly proportional to the model size, which can be very large for modern deep neural networks and lead to severe degradation in model accuracy. On the other hand, various compression techniques have been proposed to improve the communication efficiency of federated learning, but their interplay with privacy protection is largely ignored. Motivated by the observation that privacy protection and communication reduction are closely related in the context of FL, we propose a new federated learning scheme called CMP-Fed that achieves agent-level differential privacy with high model accuracy by leveraging the communication compression techniques in FL with large model sizes. The key component of CMP-Fed is compressed model perturbation (CMP), which first compresses the shared model updates before perturbing them with random noise at each communication round of federated learning. Experimental results based on Fashion-MNIST dataset show that CMP-Fed can largely outperform the existing differentially private federated learning schemes in terms of model accuracy under the same privacy guarantee while still enjoying the communication benefit of model compression.