DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answering
DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answering
复制标题
DisentQA：通过反事实问答来解开参数和上下文知识
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Omri Abend
中科院分区：
文献类型：
--
作者：
Ella Neeman;Roee Aharoni;Or Honovich;Leshem Choshen;Idan Szpektor;Omri Abend
问答模型在推理期间通常可以访问两个“知识”来源：（1）参数知识 - 模型权重中编码的事实知识；（2）上下文知识 - 给予模型以生成有根据的答案的外部知识（例如维基百科段落）。将这两种知识来源纠缠在一起是生成式问答模型的核心问题，因为尚不清楚答案是否源于给定的非参数知识。这种不明确性会对信任、可解释性和事实性问题产生影响。在这项工作中，我们提出了一种新的范式，其中训练 QA 模型来理清这两种知识来源。使用反事实数据增强，我们引入了一种模型，可以预测给定问题的两个答案：一个基于给定的上下文知识，一个基于参数知识。我们在自然问题数据集上的实验表明，这种方法提高了 QA 模型的性能，使它们对两个知识源之间的知识冲突更加鲁棒，同时生成有用的解开答案。
Question answering models commonly have access to two sources of “knowledge” during inference time: (1) parametric knowledge - the factual knowledge encoded in the model weights, and (2) contextual knowledge - external knowledge (e.g., a Wikipedia passage) given to the model to generate a grounded answer. Having these two sources of knowledge entangled together is a core issue for generative QA models as it is unclear whether the answer stems from the given non-parametric knowledge or not. This unclarity has implications on issues of trust, interpretability and factuality. In this work, we propose a new paradigm in which QA models are trained to disentangle the two sources of knowledge. Using counterfactual data augmentation, we introduce a model that predicts two answers for a given question: one based on given contextual knowledge and one based on parametric knowledge. Our experiments on the Natural Questions dataset show that this approach improves the performance of QA models by making them more robust to knowledge conflicts between the two knowledge sources, while generating useful disentangled answers.