Towards Debiasing NLU Models from Unknown Biases
Towards Debiasing NLU Models from Unknown Biases
复制标题
DOI：
10.18653/v1/2020.emnlp-main.613
复制
发表时间：
2020-09
期刊：
影响因子：
--
通讯作者：
Prasetya Ajie Utama;N. Moosavi;Iryna Gurevych
中科院分区：
文献类型：
--
作者：
Prasetya Ajie Utama;N. Moosavi;Iryna Gurevych
NLU模型经常利用偏差来实现高的特定于机器人的性能，而没有正确地学习预期的任务。最近提出的去偏置方法被证明是有效的，在减轻这种趋势。然而，这些方法依赖于一个主要的假设，即偏见的类型应该是先验已知的，这限制了它们在许多NLU任务和数据集上的应用。在这项工作中，我们提出了第一步，通过引入一个自我去偏框架，防止模型主要利用偏见，而不知道他们提前弥合这一差距。建议的框架是通用的，并补充现有的去偏置方法。我们表明，它允许这些现有的方法保持对挑战数据集的改进（即，设计用于暴露模型对偏差的依赖的示例集），而不是专门针对某些偏差。此外，评价表明，应用该框架可提高总体稳健性。
NLU models often exploit biases to achieve high dataset-specific performance without properly learning the intended task. Recently proposed debiasing methods are shown to be effective in mitigating this tendency. However, these methods rely on a major assumption that the types of bias should be known a-priori, which limits their application to many NLU tasks and datasets. In this work, we present the first step to bridge this gap by introducing a self-debiasing framework that prevents models from mainly utilizing biases without knowing them in advance. The proposed framework is general and complementary to the existing debiasing methods. We show that it allows these existing methods to retain the improvement on the challenge datasets (i.e., sets of examples designed to expose models' reliance on biases) without specifically targeting certain biases. Furthermore, the evaluation suggests that applying the framework results in improved overall robustness.