Distributed Learning without Distress: Privacy-Preserving Empirical Risk Minimization
Distributed Learning without Distress: Privacy-Preserving Empirical Risk Minimization
复制标题
DOI：
--
复制
发表时间：
2018-12
期刊：
影响因子：
--
通讯作者：
Bargav Jayaraman;Lingxiao Wang;David Evans;Quanquan Gu
中科院分区：
文献类型：
--
作者：
Bargav Jayaraman;Lingxiao Wang;David Evans;Quanquan Gu
分布式学习允许一组独立的数据所有者在不暴露其私人数据的情况下协作学习模型。我们提出了一种分布式学习方法，该方法将差异隐私与安全的多方计算相结合。我们探讨了两种流行的差异隐私，输出扰动和梯度扰动的方法，并在分布式学习环境中促进两种方法的最新方法。在我们的输出扰动方法中，双方将本地模型结合在安全的计算中，然后在揭示模型之前添加所需的差分隐私噪声。在我们的梯度扰动方法中，数据所有者通过迭代学习算法协作训练全球模型。在每次迭代中，各方都会在安全的计算中汇总其本地梯度，从而增加足够的噪声，以确保在显示梯度更新之前的隐私。对于这两种方法，我们都表明，通过在聚合后添加安全计算中的噪声，可以在多方设置中降低噪声，从而渐近地改善最佳先前结果。现实世界数据集的实验表明，我们的方法为典型的隐私要求提供了可观的实用性收益。
Distributed learning allows a group of independent data owners to collaboratively learn a model over their data sets without exposing their private data. We present a distributed learning approach that combines differential privacy with secure multi-party computation. We explore two popular methods of differential privacy, output perturbation and gradient perturbation, and advance the state-of-the-art for both methods in the distributed learning setting. In our output perturbation method, the parties combine local models within a secure computation and then add the required differential privacy noise before revealing the model. In our gradient perturbation method, the data owners collaboratively train a global model via an iterative learning algorithm. At each iteration, the parties aggregate their local gradients within a secure computation, adding sufficient noise to ensure privacy before the gradient updates are revealed. For both methods, we show that the noise can be reduced in the multi-party setting by adding the noise inside the secure computation after aggregation, asymptotically improving upon the best previous results. Experiments on real world data sets demonstrate that our methods provide substantial utility gains for typical privacy requirements.