PrivateFL: Accurate, Differentially Private Federated Learning via Personalized Data Transformation-猫眼课题宝

DOI：

发表时间：

2023

期刊：

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Yuchen Yang;Bo Hui;Haolin Yuan;N. Gong;Yinzhi Cao

联合学习（FL）使多个客户端能够在中央服务器的协调下协作训练模型。虽然FL通过将每个客户端的训练数据保存在本地来提高数据隐私，但攻击者-例如，一个不可信的服务器-仍然可以通过各种推理攻击危及客户端的本地训练数据的隐私。保护FL隐私的实际方法是差分隐私（DP），它在训练期间添加随机噪声。然而，当应用于FL时，DP受到一个关键的限制：它实质上牺牲了模型的准确性-这比应用于传统的集中式学习更严重-以实现有意义的隐私级别。本文研究了FL+DP算法精度下降的原因，并设计了一种提高精度的方法。首先，我们提出这种准确性下降的部分原因是DP在本地训练期间添加不同的随机噪声时，在FL客户端之间引入了额外的异质性。据我们所知，我们是第一个将FL中的DP与客户端异质性联系起来的公司。其次，我们设计了P RIVATE FL来学习FL中准确的、差异化的私有模型，同时降低异质性。关键想法是在本地培训期间共同学习每个客户的差异化隐私、个性化数据转换。个性化的数据转换改变了客户端的本地数据分布，以补偿DP引入的异构性，从而提高FL模型的准确性。在评估中，我们联合收割机和比较P RIVATE FL与八个国家的最先进的差分私人FL方法在七个基准数据集，包括六个图像和一个非图像数据集。我们的研究结果表明，P RIVATE FL学习具有小ε的准确FL模型，例如，CIFAR-10上的93.3%，100个客户在（ε = 2，δ = 1 e − 3）-DP下。此外，P RIVATE FL可以与先前的工作相结合，以减少DP诱导的异质性

Federated learning (FL) enables multiple clients to collaboratively train a model with the coordination of a central server. Although FL improves data privacy via keeping each client’s training data locally, an attacker—e.g., an untrusted server— can still compromise the privacy of clients’ local training data via various inference attacks. A de facto approach to preserving FL privacy is Differential Privacy (DP), which adds random noise during training. However, when applied to FL, DP suffers from a key limitation: it sacrifices the model accuracy substantially—which is even more severely than being applied to traditional centralized learning—to achieve a meaningful level of privacy. In this paper, we study the accuracy degradation cause of FL+DP and then design an approach to improve the accuracy. First, we propose that such accuracy degradation is partially because DP introduces additional heterogeneity among FL clients when adding different random noise with clipping bias during local training. To the best of our knowledge, we are the first to associate DP in FL with client heterogeneity. Second, we design P RIVATE FL to learn accurate, differentially private models in FL with reduced heterogeneity. The key idea is to jointly learn a differentially private, personalized data transformation for each client during local training. The personalized data transformation shifts client’s local data distribution to compensate the heterogeneity introduced by DP, thus improving FL model’s accuracy. In the evaluation, we combine and compare P RIVATE FL with eight state-of-the-art differentially private FL methods on seven benchmark datasets, including six image and one non-image datasets. Our results show that P RIVATE FL learns accurate FL models with a small ε , e.g., 93.3% on CIFAR-10 with 100 clients under ( ε = 2, δ = 1 e − 3)-DP. Moreover, P RIVATE FL can be combined with prior works to reduce DP-induced heterogeneity

PrivateFL: Accurate, Differentially Private Federated Learning via Personalized Data Transformation

文献摘要

相似文献