Entity-Based Knowledge Conflicts in Question Answering
Entity-Based Knowledge Conflicts in Question Answering
复制标题
DOI：
10.18653/v1/2021.emnlp-main.565
复制
发表时间：
2021-09
期刊：
影响因子：
--
通讯作者：
S. Longpre;Kartik Perisetla;Anthony Chen;Nikhil Ramesh;Chris DuBois;Sameer Singh
中科院分区：
文献类型：
--
作者：
S. Longpre;Kartik Perisetla;Anthony Chen;Nikhil Ramesh;Chris DuBois;Sameer Singh
知识依赖型任务通常使用两种知识来源：参数，在训练时学习，以及上下文，在推理时作为段落给出。为了理解模型如何一起使用这些资源，我们将知识冲突问题形式化，即上下文信息与学习信息相矛盾。通过分析流行模型的行为，我们测量了它们对记忆信息的过度依赖（幻觉的原因），并发现了加剧这种行为的重要因素。最后，我们提出了一种简单的方法来减轻对参数知识的过度依赖，最大限度地减少幻觉，并将分布外泛化提高了4% -7%。我们的研究结果证明了从业者评估模型倾向于产生幻觉而不是阅读的重要性，并表明我们的缓解策略鼓励泛化到不断变化的信息（即时间相关查询）。为了鼓励这些实践，我们发布了生成知识冲突的框架。
Knowledge-dependent tasks typically use two sources of knowledge: parametric, learned at training time, and contextual, given as a passage at inference time. To understand how models use these sources together, we formalize the problem of knowledge conflicts, where the contextual information contradicts the learned information. Analyzing the behaviour of popular models, we measure their over-reliance on memorized information (the cause of hallucinations), and uncover important factors that exacerbate this behaviour. Lastly, we propose a simple method to mitigate over-reliance on parametric knowledge, which minimizes hallucination, and improves out-of-distribution generalization by 4% - 7%. Our findings demonstrate the importance for practitioners to evaluate model tendency to hallucinate rather than read, and show that our mitigation strategy encourages generalization to evolving information (i.e. time-dependent queries). To encourage these practices, we have released our framework for generating knowledge conflicts.