Removing Disparate Impact on Model Accuracy in Differentially Private Stochastic Gradient Descent
Removing Disparate Impact on Model Accuracy in Differentially Private Stochastic Gradient Descent
复制标题
DOI：
10.1145/3447548.3467268
复制
发表时间：
2021-08
期刊：
影响因子：
--
通讯作者：
Depeng Xu;Wei Du;Xintao Wu
中科院分区：
文献类型：
--
作者：
Depeng Xu;Wei Du;Xintao Wu
在差分私有随机梯度下降（DPSGD）中，梯度裁剪和随机噪声添加不成比例地影响代表性不足和复杂的类和子群。因此，DPSGD具有不同的影响：与原始的非私有模型相比，使用DPSGD训练的模型在这些类和子组上的准确性往往会下降得更多。如果原始模型是不公平的，因为它的准确性在所有子组中都不一样，DPSGD加剧了这种不公平。在这项工作中，我们研究了由于差分隐私，比较预测精度w.r.t.的变化效用损失的不平等。每个组在私有模型和非私有模型之间。我们分析了w.r.t.隐私的成本。并解释组样本大小沿着其他因素如何与隐私对组准确性的影响相关。此外，我们提出了一种改进的DPSGD算法，称为DPSGD-F，以实现差分隐私，差分隐私的平等成本，以及良好的实用性。DPSGD-F根据组裁剪偏差自适应地调整组中样本的贡献，使得差分隐私对组准确性没有不同的影响。我们的实验评估表明，我们的删除算法的有效性，实现了令人满意的效用的差异隐私的相等成本。
In differentially private stochastic gradient descent (DPSGD), gradient clipping and random noise addition disproportionately affect underrepresented and complex classes and subgroups. As a consequence, DPSGD has disparate impact: the accuracy of a model trained using DPSGD tends to decrease more on these classes and subgroups vs. the original, non-private model. If the original model is unfair in the sense that its accuracy is not the same across all subgroups, DPSGD exacerbates this unfairness. In this work, we study the inequality in utility loss due to differential privacy, which compares the changes in prediction accuracy w.r.t. each group between the private model and the non-private model. We analyze the cost of privacy w.r.t. each group and explain how the group sample size along with other factors is related to the privacy impact on group accuracy. Furthermore, we propose a modified DPSGD algorithm, called DPSGD-F, to achieve differential privacy, equal costs of differential privacy, and good utility. DPSGD-F adaptively adjusts the contribution of samples in a group depending on the group clipping bias such that differential privacy has no disparate impact on group accuracy. Our experimental evaluation shows the effectiveness of our removal algorithm on achieving equal costs of differential privacy with satisfactory utility.