Canary Extraction in Natural Language Understanding Models
Canary Extraction in Natural Language Understanding Models
复制标题
自然语言理解模型中的金丝雀提取
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Rahul Gupta
中科院分区：
文献类型：
--
作者：
Rahil Parikh;Christophe Dupuy;Rahul Gupta
自然语言理解（NLU）模型可以在敏感信息（如电话号码，邮政编码等）上进行训练。最近的文献集中在模型反演攻击（ModIvA）上，可以从模型参数中提取训练数据。在这项工作中，我们通过提取插入NLU训练数据中的金丝雀来提出这种攻击的一个版本。在攻击中，对模型具有开放式访问权限的对手重建模型训练集中包含的金丝雀。我们评估我们的方法，通过执行文本完成金丝雀，并证明，通过使用前缀（非敏感）令牌的金丝雀，我们可以生成完整的金丝雀。例如，我们的攻击能够在NLU模型的训练数据集中重建四位数代码，其最佳配置的概率为0.5。作为对策，我们确定了几种防御机制，当结合起来，有效地消除了ModIvA在我们的实验中的风险。
Natural Language Understanding (NLU) models can be trained on sensitive information such as phone numbers, zip-codes etc. Recent literature has focused on Model Inversion Attacks (ModIvA) that can extract training data from model parameters. In this work, we present a version of such an attack by extracting canaries inserted in NLU training data. In the attack, an adversary with open-box access to the model reconstructs the canaries contained in the model’s training set. We evaluate our approach by performing text completion on canaries and demonstrate that by using the prefix (non-sensitive) tokens of the canary, we can generate the full canary. As an example, our attack is able to reconstruct a four digit code in the training dataset of the NLU model with a probability of 0.5 in its best configuration. As countermeasures, we identify several defense mechanisms that, when combined, effectively eliminate the risk of ModIvA in our experiments.
DOI：
10.1145/3292500.3330885
发表时间：
2018-11
期刊：
Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
影响因子：
--
作者：
Congzheng Song;Vitaly Shmatikov
通讯作者：
Congzheng Song;Vitaly Shmatikov