Neural-Symbolic Reasoning for the Verification of Complex Claims

本博士学位侧重于复杂文本声明的自动验证。这项任务也被称为事实核查，传统上在公司或新闻行业都是人工进行的。社交媒体的最新发展使人们能够从更多不同的来源分发信息。在这样一个动态的环境中传播未经核实的信息对错误信息提出了挑战，并已经影响了世界的社会政治格局。人工事实核查非常耗时，似乎自然不适合解决这个问题。因此，对索赔的自动核查表现出越来越大的兴趣。然而，尽管大多数现实世界的索赔需要更复杂的推理，但目前的事实核查方法只关注非常简单和简短的索赔。他们将这项任务简化为文本蕴涵(Thorne和Vlachos，2018)，直接将索赔与潜在证据进行比较。然而，为了正确评估大多数复杂索赔的真实性，有必要汇总和组合关于多个答案和来源的信息。此外，这些模型的决策解释是极具挑战性的。最初的目的是探索为评估复杂索赔的真实性而进行了大量研究的问题回答任务。这种方法由三个步骤组成。首先，如(Vlachos和Riedel，2014)所述，通过提出适当的问题将索赔分解为更小的部分，这些问题都在答复索赔中发挥作用。其次，使用包含知识库的问答模型(QA)为所提出的问题生成可解释的答案。最后，对每个问题的答案进行汇总和组合，以正确评估真实性。例如，鉴于“糖尿病患者根据患者的年龄有不同的治疗计划”这一说法，对诸如“存在哪些类型的糖尿病”或“糖尿病的治疗计划是什么样子的”等问题的回答？创建中间答案以评估复杂索赔本身。为了解决针对给定索赔生成可回答问题的任务，研究将重点放在神经序列到序列模型(Sutskever等人，2014)。为给定的声明找到最合适的问题会创建一个新任务，因为每个生成的问题都应该与添加新知识的答案相关。解决这一任务的第一步将是创建一个数据集，以实现直接监督。考虑到根据不同的事实核查项目选择的非平凡索赔和各自答案的数据集，可以手动创建一组相关问题，为特定索赔的分项索赔提供答案。通过使用众包和专用接口来加快这一过程，创建一个包含数千个样本的数据集似乎是合理的。要回答这些产生的问题，需要复杂的质量保证系统。为了将推理过程整合到这些模型中，我们的目标是使用多跳QA作为我的起点，在这种情况下，需要将多个文档组合在一起才能得到正确的答案。为了进一步将结构化知识纳入知识库的形式，我们将探索神经符号推理(d‘Avila Garcez等人，2002)，其目的是将显式推理和神经学习方法相结合，以产生对查询/输入三元组的可解释答案。这将是一个巨大的挑战，使这些模型能够处理自然语言输入。然后，我们希望也使用神经符号学习来组合每个问题的答案，以对最初的主张提供可解释的最终裁决。这种高度可解释的方法将有效地检测模型偏差，然后我们打算利用这些偏差。除了它与计算新闻学的相关性外，这项工作的结果有望增加其他领域的真实性，如在科学出版物、(保险)合同和医疗记录中。

This PhD focuses on the automatic verification of complex textual claims. This task, also called fact-checking, has traditionally been conducted manually in companies, or in the news industry. Recent developments of social media enable the distribution of information from a much larger variety of sources. Distributing unverified information in such a dynamic environment poses a challenge for misinformation and has already affected the world's socio-political landscape. Manual fact-checking is very time-intensive and seems naturally unfit to solve this issue. Thus, increased interest is shown to the automated verification of claims. Yet, while most real-world claims require more complex reasoning, current fact-checking approaches focus only on very simple and short claims. They reduce the task to one of textual entailment (Thorne and Vlachos, 2018), directly comparing the claim with potential evidence. However, to assess the truthfulness of most complex claims correctly, it is necessary to aggregate and combine information about multiple answers and sources. Moreover, the interpretation of these models' decisions is extremely challenging.The initial aim is to explore the heavily researched task of question answering for assessing the veracity of complex claims. This approach consists of three steps. First, decomposing the claim into smaller pieces by formulating appropriate questions that all play part in answering the claim, as described in (Vlachos and Riedel, 2014). Secondly, question answering models (QA) with incorporated knowledge bases are employed to generate interpretable answers for the posed questions. Finally, the answers to each question are aggregated and combined to assess the truthfulness correctly. For instance, given the claim "Patients with diabetes have different treatment plans depending on the patient's age", answers to questions such as "What types of diabetes exist" or "How does a treatment plan for diabetes look like?" create intermediate answers to assess the complex claim itself.To tackle the task of generating answerable questions to a given claim, the research will focus on the neural sequence to sequence models (Sutskever et al., 2014). Finding the most suitable questions for a given claim creates a new task as each generated question should relate to answers that add new knowledge. The first step to solving this task will be the creation of a dataset to enable direct supervision. Given a dataset of non-trivial claims and respective answers, selected on the basis of different fact-checking projects, a set of relevant questions to provide answers for sub-claims of a given claim can be manually created. By using crowd-sourcing and a dedicated interface to speed up the process, the creation of a dataset with several thousand samples appears reasonable.To answer these generated questions, sophisticated QA systems are needed. To incorporate reasoning processes in these models we aim to use multi-hop QA, where multiple documents are required to be combined to reach the correct answer, as my starting point. To further incorporate structured knowledge in form of knowledge bases, we are to explore neural-symbolic reasoning (d'Avila Garcez et al., 2002), which aims to combine both explicit reasoning and neural learning methods to create interpretable answers to queries/input triples. It will be a great challenge to make these models able to handle natural language input.We then hope to also use neural-symbolic learning to combine the answers for each question to provide an interpretable final verdict on the initial claim. This highly interpretable approach will be efficient in detecting model biases which we then aim to explore.In addition to its relevance to computational journalism, the results of the work will hopefully increase factuality in other areas such as in scientific publications, in (insurance) contracts, and in medical records.