Auditing Data Provenance in Text-Generation Models
Auditing Data Provenance in Text-Generation Models
复制标题
DOI：
10.1145/3292500.3330885
复制
发表时间：
2018-11
期刊：
影响因子：
--
通讯作者：
Congzheng Song;Vitaly Shmatikov
中科院分区：
文献类型：
--
作者：
Congzheng Song;Vitaly Shmatikov
为了帮助执行数据保护法规，例如GDPR并检测未经授权的个人数据使用，我们开发了一种新的模型审核技术，可帮助用户检查他们的数据是否用于培训机器学习模型。我们专注于审核生成自然语言文本的深度学习模型，包括单词预测和对话生成。这些模型是流行的在线服务的核心，经常接受对个人数据（例如用户的消息，搜索，聊天和评论）进行培训。如果使用特定用户的文本训练它（在其他成千上万的用户中），我们设计和评估了一种可以检测到模型的黑盒审计方法，该方法几乎没有疑问。我们从经验上表明，我们的方法可以成功地审核不适合培训数据的通用模型。我们还分析了文本生成模型如何记住单词序列，并解释了为什么这种记忆使它们可以接受审计。
To help enforce data-protection regulations such as GDPR and detect unauthorized uses of personal data, we develop a new model auditing technique that helps users check if their data was used to train a machine learning model. We focus on auditing deep-learning models that generate natural-language text, including word prediction and dialog generation. These models are at the core of popular online services and are often trained on personal data such as users' messages, searches, chats, and comments. We design and evaluate a black-box auditing method that can detect, with very few queries to a model, if a particular user's texts were used to train it (among thousands of other users). We empirically show that our method can successfully audit well-generalized models that are not overfitted to the training data. We also analyze how text-generation models memorize word sequences and explain why this memorization makes them amenable to auditing.