Renyi Differentially Private ERM for Smooth Objectives
Renyi Differentially Private ERM for Smooth Objectives
复制标题
DOI：
--
复制
发表时间：
2019-04
期刊：
影响因子：
--
通讯作者：
Chen-Chen-Chen;Jaewoo Lee;Daniel Kifer
中科院分区：
文献类型：
--
作者：
Chen-Chen-Chen;Jaewoo Lee;Daniel Kifer
本文提出了一种求解凸经验风险最小化问题的Renyi差分私有随机梯度下降算法。该算法使用输出扰动，并利用SGD内部的随机性，这创建了一种“随机化的敏感性”，以减少添加的噪声量。输出扰动的好处之一是，我们可以加入周期性平均步骤，以进一步降低灵敏度，同时提高精度(减少众所周知的SGD在最佳附近的振荡行为)。Renyi Differential Privacy可以用来提供(epsilon，Delta)-Differential隐私保证，从而提供与以前工作的比较。实验结果表明，该方法在差分私有ERM上的性能优于已有的方法。
In this paper, we present a Renyi Differentially Private stochastic gradient descent (SGD) algorithm for convex empirical risk minimization. The algorithm uses output perturbation and leverages randomness inside SGD, which creates a "randomized sensitivity", in order to reduce the amount of noise that is added. One of the benefits of output perturbation is that we can incorporate a periodic averaging step that serves to further reduce sensitivity while improving accuracy (reducing the well-known oscillating behavior of SGD near the optimum). Renyi Differential Privacy can be used to provide (epsilon, delta)-differential privacy guarantees and hence provide a comparison with prior work. An empirical evaluation demonstrates that the proposed method outperforms prior methods on differentially private ERM.