Overfitting, robustness, and malicious algorithms: A study of potential causes of privacy risk in machine learning
Overfitting, robustness, and malicious algorithms: A study of potential causes of privacy risk in machine learning
复制标题
DOI：
10.3233/jcs-191362
复制
发表时间：
2020-02
期刊：
影响因子：
--
通讯作者：
Samuel Yeom;Irene Giacomelli;Alan Menaged;Matt Fredrikson;S. Jha
中科院分区：
文献类型：
--
作者：
Samuel Yeom;Irene Giacomelli;Alan Menaged;Matt Fredrikson;S. Jha
。机器学习算法在应用于敏感数据时，对隐私构成了明显的威胁。越来越多的以前的工作表明，由这些算法产生的模型可能会通过模型的结构或其可观察到的行为将训练数据中的特定fic私人信息泄露给攻击者。本文研究了允许训练集成员资格推理攻击者或属性推理攻击者学习此类信息的因素。使用形式分析和经验分析，我们说明了这些因素与几种流行的机器学习算法中出现的隐私风险之间的明确关系。我们fi发现Overfiting是充分的，允许攻击者执行成员关系推理，并且当目标属性满足与其相关的某些条件时，就可以进行属性推理攻击。我们还探讨了成员关系推理和属性推理之间的联系，表明了两者之间存在着深刻的联系，从而导致了有效的新攻击。我们证明了这些攻击不需要过度fi设置，证明了其他因素，如对范数有界输入扰动的健壮性和恶意训练算法，也可以显著增加隐私风险。值得注意的是，由于健壮性旨在防御对模型预测完整性的攻击，这些结果表明在某些情况下可能很难同时防御隐私和完整性攻击。
. Machine learning algorithms, when applied to sensitive data, pose a distinct threat to privacy. A growing body of prior work demonstrates that models produced by these algorithms may leak specific private information in the training data to an attacker, either through the models’ structure or their observable behavior. This article examines the factors that can allow a training set membership inference attacker or an attribute inference attacker to learn such information. Using both formal and empirical analyses, we illustrate a clear relationship between these factors and the privacy risk that arises in several popular machine learning algorithms. We find that overfitting is sufficient to allow an attacker to perform membership inference and, when the target attribute meets certain conditions about its influence, attribute inference attacks. We also explore the connection between membership inference and attribute inference, showing that there are deep connections between the two that lead to effective new attacks. We show that overfitting is not necessary for these attacks, demonstrating that other factors, such as robustness to norm-bounded input perturbations and malicious training algorithms, can also significantly increase the privacy risk. Notably, as robustness is intended to be a defense against attacks on the integrity of model predictions, these results suggest it may be difficult in some cases to simultaneously defend against privacy and integrity attacks.