Robustness to Programmable String Transformations via Augmented Abstract Training
Robustness to Programmable String Transformations via Augmented Abstract Training
复制标题
DOI：
--
复制
发表时间：
2020-02
期刊：
影响因子：
--
通讯作者：
Yuhao Zhang;Aws Albarghouthi;Loris D'antoni
中科院分区：
文献类型：
--
作者：
Yuhao Zhang;Aws Albarghouthi;Loris D'antoni
用于自然语言处理任务的深度神经网络容易受到对抗性输入扰动的影响。在本文中，我们提出了一种通用语言，用于以编程方式指定字符串转换——例如，插入、删除、替换、交换等——与手头的任务相关。然后，我们提出了一种对抗性训练模型的方法，该模型对这种用户定义的字符串转换具有鲁棒性。我们的方法结合了基于搜索的对抗性训练技术和基于抽象的技术的优点。具体来说，我们将展示如何将一组用户定义的字符串转换分解为两个组件规范，一个受益于搜索，另一个受益于抽象。我们使用我们的技术在AG和SST2数据集上训练模型，并表明所得到的模型对于模仿拼写错误的用户定义转换和其他意义保留转换的组合具有鲁棒性。
Deep neural networks for natural language processing tasks are vulnerable to adversarial input perturbations. In this paper, we present a versatile language for programmatically specifying string transformations -- e.g., insertions, deletions, substitutions, swaps, etc. -- that are relevant to the task at hand. We then present an approach to adversarially training models that are robust to such user-defined string transformations. Our approach combines the advantages of search-based techniques for adversarial training with abstraction-based techniques. Specifically, we show how to decompose a set of user-defined string transformations into two component specifications, one that benefits from search and another from abstraction. We use our technique to train models on the AG and SST2 datasets and show that the resulting models are robust to combinations of user-defined transformations mimicking spelling mistakes and other meaning-preserving transformations.