Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence
Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence
复制标题
DOI：
10.18653/v1/2021.naacl-main.52
复制
发表时间：
2021-03
期刊：
影响因子：
--
通讯作者：
Tal Schuster;Adam Fisch;R. Barzilay
中科院分区：
文献类型：
--
作者：
Tal Schuster;Adam Fisch;R. Barzilay
典型的验证模型使用书面证据来验证索赔，但是，随着时间的流逝，随着时间的流逝和修订，模型必须敏感。基于挑战案例的基准，需要识别事实验证模型，并适应略微变化。修改一个基本事实，并利用这些修订以及其他合成构造的修订，以创建超过400,000个索赔证据对，与以前的资源不同。在语言和内容上，除了一个人支持给定的主张，而另一个人则表明，使用这种设计培训会增加鲁棒性 - 提高训练准确性在对抗事实验证中的准确性为10％，另外6％在对抗性的自然语言推论（NLI）中，维生素的结构使我们定义了对事实检查资源的其他任务：在证据中标记相关词事实修订，并通过事实一致的文本生成提供自动编辑。
Typical fact verification models use retrieved written evidence to verify claims. Evidence sources, however, often change over time as more information is gathered and revised. In order to adapt, models must be sensitive to subtle differences in supporting evidence. We present VitaminC, a benchmark infused with challenging cases that require fact verification models to discern and adjust to slight factual changes. We collect over 100,000 Wikipedia revisions that modify an underlying fact, and leverage these revisions, together with additional synthetically constructed ones, to create a total of over 400,000 claim-evidence pairs. Unlike previous resources, the examples in VitaminC are contrastive, i.e., they contain evidence pairs that are nearly identical in language and content, with the exception that one supports a given claim while the other does not. We show that training using this design increases robustness—improving accuracy by 10% on adversarial fact verification and 6% on adversarial natural language inference (NLI). Moreover, the structure of VitaminC leads us to define additional tasks for fact-checking resources: tagging relevant words in the evidence for verifying the claim, identifying factual revisions, and providing automatic edits via factually consistent text generation.