Stochastic ADMM Based Distributed Machine Learning with Differential Privacy
Stochastic ADMM Based Distributed Machine Learning with Differential Privacy
复制标题
基于随机ADMM的差分隐私分布式机器学习
DOI：
10.1007/978-3-030-37228-6_13
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Han, Zhu
中科院分区：
文献类型：
--
作者：
Ding, Jiahao;Errapotu, Sai Mounika;Zhang, Haijun;Gong, Yanmin;Pan, Miao;Han, Zhu
在大数据时代采用各种机器学习技术做出有效决策的同时，保护敏感数据的隐私构成了重大挑战。本文提出了一种隐私保护的分布式机器学习算法来解决这个问题。假设每个数据提供者拥有一个不同样本大小的数据集，我们的目标是在不泄露数据样本的任何敏感信息的情况下，以分布式的方式在所有局部数据集的并集上学习一个通用的分类器。这样的算法需要联合考虑高效的分布式学习和有效的隐私保护。在该算法中，我们将随机交替方向乘子法(ADMM)扩展到分布式环境中进行分布式学习。为了在迭代过程中保护隐私，我们将差分隐私和随机ADMM结合起来。特别地，我们提出了一种新的基于随机ADMM的隐私保护分布式机器学习(PS-ADMM)算法，该算法通过扰动更新梯度来提供差分隐私保证并且具有较低的计算代价。我们从理论上证明了我们提出的PS-ADMM在强凸目标下的收敛速度和效用界。通过在真实数据集上的实验，我们证明了PS-ADMM在相同的差分隐私保证下比其他差分私有ADMM算法有更好的性能。
While embracing various machine learning techniques to make effective decisions in the big data era, preserving the privacy of sensitive data poses significant challenges. In this paper, we develop a privacy-preserving distributed machine learning algorithm to address this issue. Given the assumption that each data provider owns a dataset with different sample size, our goal is to learn a common classifier over the union of all the local datasets in a distributed way without leaking any sensitive information of the data samples. Such an algorithm needs to jointly consider efficient distributed learning and effective privacy preservation. In the proposed algorithm, we extend stochastic alternating direction method of multipliers (ADMM) in a distributed setting to do distributed learning. For preserving privacy during the iterative process, we combine differential privacy and stochastic ADMM together. In particular, we propose a novel stochastic ADMM based privacy-preserving distributed machine learning (PS-ADMM) algorithm by perturbing the updating gradients, that provide differential privacy guarantee and have a low computational cost. We theoretically demonstrate the convergence rate and utility bound of our proposed PS-ADMM under strongly convex objective. Through our experiments performed on real-world datasets, we show that PS-ADMM outperforms other differentially private ADMM algorithms under the same differential privacy guarantee.
DOI：
--
发表时间：
2018-02
期刊：
ArXiv
影响因子：
--
作者：
Di Wang;Minwei Ye;Jinhui Xu
通讯作者：
Di Wang;Minwei Ye;Jinhui Xu
具有差异隐私的物联网中群体感知的实用协作学习
DOI：
--
发表时间：
2018
期刊：
IEEE Conference on Communications and Network Security
影响因子：
--
作者：
Yuanxiong Guo;Yanmin Gong
通讯作者：
Yanmin Gong