Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Models

Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Models
复制标题

DOI：
10.48550/arxiv.2310.05253
复制
发表时间：
2023-10
期刊：
ArXiv
影响因子：
--
通讯作者：
Haoran Wang;Kai Shu
Haoran Wang;Kai Shu
中科院分区：
其他
文献类型：
--
作者：
Haoran Wang;Kai Shu

文献摘要

相似文献

声明验证在打击错误信息方面发挥着至关重要的作用。虽然现有的索赔验证工作已经显示出有希望的结果，但仍未解决的一个关键难题是了解如何在不依赖人工注释数据的情况下验证索赔，而大规模创建人工注释数据的成本很高。此外，模型必须提供全面的解释，以证明其决策的合理性并协助人类事实检查人员。本文提出了一阶逻辑引导的知识基础 (FOLK) 推理，可以验证复杂的主张并生成解释，而无需使用大型语言模型 (LLM) 带注释的证据。 FOLK 利用法学硕士的上下文学习能力将声明转换为由谓词组成的一阶逻辑 (FOL) 子句，每个谓词对应于需要验证的子声明。然后，FOLK 对一组基于知识的问答对执行 FOL 引导推理，以做出准确性预测并生成解释以证明其决策过程的合理性。这个过程使我们的模型具有高度解释性，以人类可读的形式对其推理过程提供清晰的解释。我们的实验结果表明，FOLK 在涵盖各种声明验证挑战的三个数据集上的表现优于强大的基线。我们的代码和数据可用。
Claim verification plays a crucial role in combating misinformation. While existing works on claim verification have shown promising results, a crucial piece of the puzzle that remains unsolved is to understand how to verify claims without relying on human-annotated data, which is expensive to create at a large scale. Additionally, it is important for models to provide comprehensive explanations that can justify their decisions and assist human fact-checkers. This paper presents First-Order-Logic-Guided Knowledge-Grounded (FOLK) Reasoning that can verify complex claims and generate explanations without the need for annotated evidence using Large Language Models (LLMs). FOLK leverages the in-context learning ability of LLMs to translate the claim into a First-Order-Logic (FOL) clause consisting of predicates, each corresponding to a sub-claim that needs to be verified. Then, FOLK performs FOL-Guided reasoning over a set of knowledge-grounded question-and-answer pairs to make veracity predictions and generate explanations to justify its decision-making process. This process makes our model highly explanatory, providing clear explanations of its reasoning process in human-readable form. Our experiment results indicate that FOLK outperforms strong baselines on three datasets encompassing various claim verification challenges. Our code and data are available.