High-accuracy low-cost privacy-preserving federated learning in IoT systems via adaptive perturbation
High-accuracy low-cost privacy-preserving federated learning in IoT systems via adaptive perturbation
复制标题
DOI：
10.1016/j.jisa.2022.103309
复制
发表时间：
2022-11
期刊：
影响因子：
--
通讯作者：
Tian Liu;Xueyang Hu;Hang Xu;Tao Shu;Diep N. Nguyen
中科院分区：
文献类型：
--
作者：
Tian Liu;Xueyang Hu;Hang Xu;Tao Shu;Diep N. Nguyen
随着物联网（IoT）的快速发展，联邦学习（FL）已被广泛应用于从收集的数据中获取见解，同时保护数据隐私。差分隐私（DP）是一种可加性噪声方案，作为FL的隐私保护方法得到了广泛的研究。然而，DP下的隐私保护通常是以牺牲底层FL过程的模型精度为代价的。在本文中，我们提出了一种新的低成本（通信和计算开销）自适应噪声扰动/屏蔽方案，以保护FL客户端的隐私而不降低全局模型的准确性。特别是，我们设置了加性噪声的大小，使其随局部模型更新的大小而自适应变化。然后，采用一种基于方向的滤波方案来加快FL模型的收敛速度。利用中心极限定理（CLT）导出了局部客户端的最大可容忍噪声边界。设计的噪声在最大程度上保护了客户端的隐私，同时保持了FL模型的准确性和收敛速度，这是由于在服务器上进行聚合操作后，噪声被抵消并形成了更集中的分布。我们从理论上证明了采用噪声扰动格式的FL与采用SGD的非私有FL保持相同的精度和收敛速度(对于凸损失函数为O （1/T)，对于非凸损失函数为O (1/T)）。我们还从收敛行为、计算效率和隐私保护方面评估了所提出方案的性能，以防止对现实世界数据集进行最先进的隐私推断攻击。实验结果表明，在客户端退出和非客户端退出情况下，采用我们提出的扰动方案的FL在FL模型的精度和收敛速度上都优于DP。与DP相比，我们的方案不会产生额外的计算和通信开销。在相同的全局模型精度下，我们的方法在防御隐私攻击方面提供了与dp相当或更好的有效性。
With the rapid development of the Internet of Things (IoT), federated learning (FL) has been widely used to obtain insights from collected data while preserving data privacy. Differential privacy (DP) is an additive noise scheme that has been widely studied as a privacy-preserving approach on FL. However, privacy protection under DP usually comes at the cost of model accuracy for the underlying FL process. In this paper, we propose a novel low-cost (for both communication and computational overhead) adaptive noise perturbation/masking scheme to protect FL clients’ privacy without degrading the global model accuracy. In particular, we set the magnitude of the additive noise to adaptively change with the magnitude of the local model updates. Then, a direction-based filtering scheme is used to accelerate the convergence of the FL model. A maximum tolerable noise bound for local clients is derived using the central limit theorem (CLT). The designed noise maximizes privacy protection for clients while preserving the accuracy and convergence rate of the FL model, as a result of the noise cancelling out and forming a more concentrated distribution after the aggregation operation on the server. We theoretically prove that FL with the proposed noise perturbation scheme retains the same accuracy and convergence rate (O (1/T) for convex loss functions and O (1/T) for non-convex loss functions) as that of non-private FL with SGD. We also evaluate the performance of the proposed scheme in terms of convergence behavior, computational efficiency, and privacy protection against state-of-the-art privacy inference attacks on real-world datasets. Experimental results show that FL with our proposed perturbation scheme outperforms DP in the accuracy and convergence rate of the FL model in both client dropout and non-client dropout scenarios. Compared with DP, our proposed scheme does not incur additional computational and communication overhead. Our approach provides DP-comparable or better effectiveness in defending against privacy attacks under the same global model accuracy.