Machine Learning Models that Remember Too Much
Machine Learning Models that Remember Too Much
复制标题
DOI：
10.1145/3133956.3134077
复制
发表时间：
2017-09
期刊：
影响因子：
--
通讯作者：
Congzheng Song;Thomas Ristenpart;Vitaly Shmatikov
中科院分区：
文献类型：
--
作者：
Congzheng Song;Thomas Ristenpart;Vitaly Shmatikov
机器学习（ML）正在成为一种商品。许多机器学习框架和服务可供那些不是机器学习专家但希望在自己的数据上训练预测模型的数据持有者使用。在敏感输入（例如个人图像或文档）上训练的机器学习模型不泄露过多关于训练数据的信息是很重要的。我们考虑一个恶意的机器学习提供者，它向数据持有者提供模型训练代码，不观察训练过程，但随后获得对所得到模型的白盒或黑盒访问权限。在这种情况下，我们设计并实现实用算法，其中一些与标准机器学习技术（如正则化和数据增强）非常相似，这些算法在模型中“记忆”关于训练数据集的信息——然而该模型与常规训练的模型一样准确和具有预测性。然后我们解释对手如何从模型中提取记忆的信息。我们在用于图像分类（CIFAR10）、人脸识别（LFW和FaceScrub）以及文本分析（20 Newsgroups和IMDB）的标准机器学习任务上评估我们的技术。在所有情况下，我们展示了我们的算法如何创建具有高预测能力但又能准确提取其训练数据子集的模型。
Machine learning (ML) is becoming a commodity. Numerous ML frameworks and services are available to data holders who are not ML experts but want to train predictive models on their data. It is important that ML models trained on sensitive inputs (e.g., personal images or documents) not leak too much information about the training data. We consider a malicious ML provider who supplies model-training code to the data holder, does \emph{not} observe the training, but then obtains white- or black-box access to the resulting model. In this setting, we design and implement practical algorithms, some of them very similar to standard ML techniques such as regularization and data augmentation, that "memorize" information about the training dataset in the model\textemdash yet the model is as accurate and predictive as a conventionally trained model. We then explain how the adversary can extract memorized information from the model. We evaluate our techniques on standard ML tasks for image classification (CIFAR10), face recognition (LFW and FaceScrub), and text analysis (20 Newsgroups and IMDB). In all cases, we show how our algorithms create models that have high predictive power yet allow accurate extraction of subsets of their training data.