Improving the Robustness of Question Answering Systems to Question Paraphrasing
Improving the Robustness of Question Answering Systems to Question Paraphrasing
复制标题
DOI：
10.18653/v1/p19-1610
复制
发表时间：
2019-07
期刊：
影响因子：
--
通讯作者：
Wee Chung Gan;H. Ng
中科院分区：
文献类型：
--
作者：
Wee Chung Gan;H. Ng
尽管提出了问题的进步（QA）系统和持有测试集的快速改进，但我们的概括性是一个令人关注的话题第一个测试集中的问题与旨在测试QA模型过敏的原始问题非常相似，而第二个测试集中的问题是在试图混淆质量检查模型的情况下，使用不正确的答案候选人的上下文单词进行解释。给定来源问题和一组释义建议的多个释义问题，我们提出了一种数据增强方法，不需要人类干预即可重新培训模型提高了质疑释义的鲁棒性。
Despite the advancement of question answering (QA) systems and rapid improvements on held-out test sets, their generalizability is a topic of concern. We explore the robustness of QA models to question paraphrasing by creating two test sets consisting of paraphrased SQuAD questions. Paraphrased questions from the first test set are very similar to the original questions designed to test QA models’ over-sensitivity, while questions from the second test set are paraphrased using context words near an incorrect answer candidate in an attempt to confuse QA models. We show that both paraphrased test sets lead to significant decrease in performance on multiple state-of-the-art QA models. Using a neural paraphrasing model trained to generate multiple paraphrased questions for a given source question and a set of paraphrase suggestions, we propose a data augmentation approach that requires no human intervention to re-train the models for improved robustness to question paraphrasing.