DP-ADMM: ADMM-Based Distributed Learning With Differential Privacy
DP-ADMM: ADMM-Based Distributed Learning With Differential Privacy
复制标题
DOI：
10.1109/tifs.2019.2931068
复制
发表时间：
2020-01-01
影响因子：
6.8
通讯作者：
Gong, Yanmin
中科院分区：
文献类型：
--
作者：
Huang, Zonghao;Hu, Rui;Gong, Yanmin
交替方向乘法器（ADMM）是一种广泛用于分布式环境中机器学习的工具，其中机器学习模型通过本地计算和消息传递的交互过程在分布式数据源上进行训练。这种迭代过程可能会引起数据所有者的隐私问题。本文的目标是为基于ADMM的分布式机器学习提供差分隐私。现有的差分隐私ADMM方法在高隐私保证下表现出低效用，并假设学习问题的目标函数是光滑和强凸的。为了解决这些问题，我们提出了一种新的差分隐私ADMM为基础的分布式学习算法称为DP-ADMM，它结合了近似增广拉格朗日函数与时变高斯噪声添加在迭代过程中，以实现更高的效用一般目标函数下相同的差分隐私保证。我们还应用矩会计方法来分析端到端的隐私损失。理论分析表明，DP-ADMM可以适用于更广泛的分布式学习问题，是可证明收敛的，并提供了一个明确的效用隐私权衡。据我们所知，这是第一篇论文，提供明确的收敛性和实用性能的差异私人ADMM为基础的分布式学习算法。评估结果表明，我们的方法可以实现良好的收敛性和模型的准确性下，高端到端的差分隐私保证。
Alternating direction method of multipliers (ADMM) is a widely used tool for machine learning in distri-buted settings where a machine learning model is trained over distributed data sources through an interactive process of local computation and message passing. Such an iterative process could cause privacy concerns of data owners. The goal of this paper is to provide differential privacy for ADMM-based distributed machine learning. Prior approaches on differentially private ADMM exhibit low utility under high privacy guarantee and assume the objective functions of the learning problems to be smooth and strongly convex. To address these concerns, we propose a novel differentially private ADMM-based distributed learning algorithm called DP-ADMM, which combines an approximate augmented Lagrangian function with time-varying Gaussian noise addition in the iterative process to achieve higher utility for general objective functions under the same differential privacy guarantee. We also apply the moments accountant method to analyze the end-to-end privacy loss. The theoretical analysis shows that the DP-ADMM can be applied to a wider class of distributed learning problems, is provably convergent, and offers an explicit utility-privacy tradeoff. To our knowledge, this is the first paper to provide explicit convergence and utility properties for differentially private ADMM-based distributed learning algorithms. The evaluation results demonstrate that our approach can achieve good convergence and model accuracy under high end-to-end differential privacy guarantee.