Explainable Multi-hop Verbal Reasoning Through Internal Monologue
Explainable Multi-hop Verbal Reasoning Through Internal Monologue
复制标题
DOI：
10.18653/v1/2021.naacl-main.97
复制
发表时间：
2021-06
期刊：
影响因子：
--
通讯作者：
Zhengzhong Liang;Steven Bethard;M. Surdeanu
中科院分区：
文献类型：
--
作者：
Zhengzhong Liang;Steven Bethard;M. Surdeanu
许多国家的最先进的（SOTA）语言模型已经取得了高精度的多跳推理问题。然而，这些方法往往是不可解释的，因为它们没有明确的中间推理步骤。此外，在简单任务上训练的模型在直接测试更复杂的问题时往往会失败。我们提出了可解释的多跳言语推理器（EVR）来解决这些限制（a）分解多跳推理问题成几个简单的，（B）使用自然语言来指导中间的推理跳。我们实现EVR扩展的经典推理范式一般问题求解器（GPS）与SOTA生成语言模型生成子目标，并在每个推理步骤中进行自然语言推理。在RuleTaker综合问题回答（QA）数据集上对EVR进行的评估表明，EVR实现了SOTA性能，同时能够以自然语言生成所有推理步骤。此外，当在更简单的任务或更少的训练数据上训练时，EVR比其他强方法更好地推广（分别高达35.7%和7.7%的绝对改进）。
Many state-of-the-art (SOTA) language models have achieved high accuracy on several multi-hop reasoning problems. However, these approaches tend to not be interpretable because they do not make the intermediate reasoning steps explicit. Moreover, models trained on simpler tasks tend to fail when directly tested on more complex problems. We propose the Explainable multi-hop Verbal Reasoner (EVR) to solve these limitations by (a) decomposing multi-hop reasoning problems into several simple ones, and (b) using natural language to guide the intermediate reasoning hops. We implement EVR by extending the classic reasoning paradigm General Problem Solver (GPS) with a SOTA generative language model to generate subgoals and perform inference in natural language at each reasoning step. Evaluation of EVR on the RuleTaker synthetic question answering (QA) dataset shows that EVR achieves SOTA performance while being able to generate all reasoning steps in natural language. Furthermore, EVR generalizes better than other strong methods when trained on simpler tasks or less training data (up to 35.7% and 7.7% absolute improvement respectively).