Differentially-Private Deep Learning from an optimization Perspective
Differentially-Private Deep Learning from an optimization Perspective
复制标题
DOI：
10.1109/infocom.2019.8737494
复制
发表时间：
2019-04
期刊：
影响因子：
--
通讯作者：
Liyao Xiang;Jingbo Yang;Baochun Li
中科院分区：
文献类型：
--
作者：
Liyao Xiang;Jingbo Yang;Baochun Li
随着用户数据众包数据挖掘量的急剧增加，对个人隐私保护的需求日益迫切。差分隐私机制通常被用于向用户数据中添加噪声，这样攻击者就无法通过从学习模型中进行推断来获得关于参与众包的个人的任何额外知识。然而，这种保护通常是在显著降低学习结果的情况下实现的。我们观察到，造成这一问题的根本原因是模型效用与数据隐私之间的关系没有准确表征，导致隐私约束过于严格。在本文中，我们从优化的角度来解决这个问题，并将问题表述为在给定一组隐私约束的情况下最小化准确性损失的问题。我们使用灵敏度来描述扰动噪声对模型效用的影响，并提出了一种新的优化的加性噪声机制，该机制在符合个人隐私约束的同时提高了整体学习精度。作为我们隐私机制的一个亮点，它在高隐私制度（当$\epsilon \rightarrow 0$）中具有高度鲁棒性，并且不受模型结构和实验设置的任何变化的影响。
With the amount of user data crowdsourced for data mining dramatically increasing, there is an urgent need to protect the privacy of individuals. Differential privacy mechanisms are conventionally adopted to add noise to the user data, so that an adversary is not able to gain any additional knowledge about individuals participating in the crowdsourcing, by inferring from the learned model. However, such protection is usually achieved with significantly degraded learning results. We have observed that the fundamental cause of this problem is that the relationship between model utility and data privacy is not accurately characterized, leading to privacy constraints that are overly strict. In this paper, we address this problem from an optimization perspective, and formulate the problem as one that minimizes the accuracy loss given a set of privacy constraints. We use sensitivity to describe the impact of perturbation noise to the model utility, and propose a new optimized additive noise mechanism that improves overall learning accuracy while conforming to individual privacy constraints. As a highlight of our privacy mechanism, it is highly robust in the high privacy regime (when $\epsilon \rightarrow 0$), and against any changes in the model structure and experimental settings.