Concealed Data Poisoning Attacks on NLP Models
Concealed Data Poisoning Attacks on NLP Models
复制标题
DOI：
10.18653/v1/2021.naacl-main.13
复制
发表时间：
2021-06
期刊：
影响因子：
--
通讯作者：
Eric Wallace;Tony Zhao;Shi Feng;Sameer Singh
中科院分区：
文献类型：
--
作者：
Eric Wallace;Tony Zhao;Shi Feng;Sameer Singh
对抗性攻击通过扰乱测试时的输入来改变自然语言处理（NLP）模型的预测。然而，对于是否以及如何通过对训练数据进行微小的、隐蔽的改变来操纵预测，人们的了解要少得多。在这项工作中，我们开发了一种新的数据投毒攻击方法，使得攻击者只要在输入中出现期望的触发短语，就能控制模型的预测。例如，我们在情感模型的训练集中插入50个投毒样本，使得当输入包含“詹姆斯·邦德”时，模型经常预测为“积极”。关键的是，我们使用基于梯度的方法来精心制作这些投毒样本，这样它们就不会提及触发短语。我们还将我们的投毒攻击应用于语言建模（“苹果iPhone”触发负面生成）和机器翻译（“冰咖啡”被误译为“热咖啡”）。最后，我们提出了三种防御方法，这些方法可以在一定程度上以预测准确性的损失或额外的人工标注为代价来减轻我们的攻击。
Adversarial attacks alter NLP model predictions by perturbing test-time inputs. However, it is much less understood whether, and how, predictions can be manipulated with small, concealed changes to the training data. In this work, we develop a new data poisoning attack that allows an adversary to control model predictions whenever a desired trigger phrase is present in the input. For instance, we insert 50 poison examples into a sentiment model’s training set that causes the model to frequently predict Positive whenever the input contains “James Bond”. Crucially, we craft these poison examples using a gradient-based procedure so that they do not mention the trigger phrase. We also apply our poison attack to language modeling (“Apple iPhone” triggers negative generations) and machine translation (“iced coffee” mistranslated as “hot coffee”). We conclude by proposing three defenses that can mitigate our attack at some cost in prediction accuracy or extra human annotation.