Retrieval-guided Counterfactual Generation for QA
Retrieval-guided Counterfactual Generation for QA
复制标题
用于 QA 的检索引导反事实生成
DOI：
--
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Ian Tenney
中科院分区：
文献类型：
--
作者：
Bhargavi Paranjape;Matthew Lamm;Ian Tenney
Deep NLP模型已被证明对输入扰动很脆弱。与世界知识，语义多样性和应对性有关。经过原始任务数据培训的模型，我们创建的反事实是流利的，语义上的多样化并自动标记的，它会通过RGF反事实来提高型号的性能开放式质量检查设置。
Deep NLP models have been shown to be brittle to input perturbations. Recent work has shown that data augmentation using counterfactuals — i.e. minimally perturbed inputs — can help ameliorate this weakness. We focus on the task of creating counterfactuals for question answering, which presents unique challenges related to world knowledge, semantic diversity, and answerability. To address these challenges, we develop a Retrieve-Generate-Filter(RGF) technique to create counterfactual evaluation and training data with minimal human supervision. Using an open-domain QA framework and question generation model trained on original task data, we create counterfactuals that are fluent, semantically diverse, and automatically labeled. Data augmentation with RGF counterfactuals improves performance on out-of-domain and challenging evaluation sets over and above existing methods, in both the reading comprehension and open-domain QA settings. Moreover, we find that RGF data leads to significant improvements in a model’s robustness to local perturbations.
DOI：
10.18653/v1/2020.findings-emnlp.117
发表时间：
2020-04
期刊：
--
影响因子：
--
作者：
Matt Gardner;Yoav Artzi;Jonathan Berant;Ben Bogin;Sihao Chen;Dheeru Dua;Yanai Elazar;Ananth Gottumukkala;Nitish Gupta;Hannaneh Hajishirzi;Gabriel Ilharco;Daniel Khashabi;Kevin Lin;Jiangming Liu;Nelson F. Liu;Phoebe Mulcaire;Qiang Ning;Sameer Singh;Noah A. Smith;Sanjay Subramanian;Eric Wallace;Ally Zhang;Ben Zhou
通讯作者：
Matt Gardner;Yoav Artzi;Jonathan Berant;Ben Bogin;Sihao Chen;Dheeru Dua;Yanai Elazar;Ananth Gottumukkala;Nitish Gupta;Hannaneh Hajishirzi;Gabriel Ilharco;Daniel Khashabi;Kevin Lin;Jiangming Liu;Nelson F. Liu;Phoebe Mulcaire;Qiang Ning;Sameer Singh;Noah A. Smith;Sanjay Subramanian;Eric Wallace;Ally Zhang;Ben Zhou