Universal Adversarial Triggers for Attacking and Analyzing NLP
Universal Adversarial Triggers for Attacking and Analyzing NLP
复制标题
DOI：
10.18653/v1/d19-1221
复制
发表时间：
2019-08
期刊：
影响因子：
--
通讯作者：
Eric Wallace;Shi Feng;Nikhil Kandpal;Matt Gardner;Sameer Singh
中科院分区：
文献类型：
--
作者：
Eric Wallace;Shi Feng;Nikhil Kandpal;Matt Gardner;Sameer Singh
对抗性示例突出显示模型漏洞，对于评估和解释很有用。超出令牌可以找到成功触发目标预测的短触发序列（例如，一个用于分类的单词和四个单词的单词）。例如，触发器会导致SNLI的准确性从89.94％降至0.55％，其中72％的“为什么”问题是“杀死美国人”的“为什么”问题，以及GPT-2语言模型即使在条件下也可以吐出种族主义的产出在非种族的上下文中，尽管使用对特定模型的白色盒子进行了优化，但它们最终会转移到其他模型中。输入敏捷，他们提供了对全球模型行为的分析。
Adversarial examples highlight model vulnerabilities and are useful for evaluation and interpretation. We define universal adversarial triggers: input-agnostic sequences of tokens that trigger a model to produce a specific prediction when concatenated to any input from a dataset. We propose a gradient-guided search over tokens which finds short trigger sequences (e.g., one word for classification and four words for language modeling) that successfully trigger the target prediction. For example, triggers cause SNLI entailment accuracy to drop from 89.94% to 0.55%, 72% of “why” questions in SQuAD to be answered “to kill american people”, and the GPT-2 language model to spew racist output even when conditioned on non-racial contexts. Furthermore, although the triggers are optimized using white-box access to a specific model, they transfer to other models for all tasks we consider. Finally, since triggers are input-agnostic, they provide an analysis of global model behavior. For instance, they confirm that SNLI models exploit dataset biases and help to diagnose heuristics learned by reading comprehension models.