RI: Small: Applying discrete reasoning steps in solving natural language processing tasks

RI：小：应用离散推理步骤解决自然语言处理任务

基本信息

批准号：
1814522
负责人：
Gregory Durrett
金额：
$ 44.76万
依托单位：
University of Texas at Austin
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2018
资助国家：
美国
起止时间：
2018-08-01 至 2022-07-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1814522&HistoricalAwards=false
关键词：
RI Small Applying discrete reasoning

项目摘要

Modern natural language processing systems are effective at shallow analysis of unstructured text data, performing tasks such as discovering events, identifying the actors of those events, and grouping events with the same actors. Neural networks help make these systems robust to effects like paraphrasing, but still capture mostly superficial text patterns. To answer deeper questions about things like causal relationships between the events in a text, a system might need to combine several pieces of information, abstract away irrelevant details, and incorporate prior world knowledge to arrive at an answer. This project aims to develop systems that can address these challenges: these systems explicitly model reasoning over text and draw on the power of neural networks to do this reasoning in a nuanced way. Such reasoning is explicitly taught to the systems via "handholding" supervision, which encourages the systems to mimic how humans solve a problem and helps them generalize better to new problem instances. This alignment with what humans do also serves to expose the systems' decision-making processes; it provides a form of explanation of their behavior so that one may evaluate them against desired criteria such as equitability.This proposal's technical innovation is focused on two fronts: designing latent variable models and exploiting new types of handholding supervision during model training. These techniques are explored in the context of three challenging problems requiring complex reasoning: (1) solving mathematical word problems; (2) resolving coreference using world knowledge; (3) answering questions from documents. For each problem, new models are proposed centering around discrete derivations of answers, which draw on state-of-the-art tools like attention-based recurrent neural networks to capture the larger context of the reasoning process. The discreteness of the models' decisions provides an anchor to incorporate auxiliary supervision, which is hard to do in fully end-to-end neural models. The nature of the handholding supervision depends on the task and is a combination of incidental supervision, heuristically identified derivations, and targeted human annotation. Each of the addressed problems tests different aspects of the approach, such as handling complex derivations and incorporating world knowledge, and these problems yield concrete evaluation frameworks to understand the efficacy of the proposed techniques.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

现代的自然语言处理系统对非结构化文本数据的浅分析有效，执行任务，例如发现事件，确定这些事件的参与者以及将事件与同一参与者分组。神经网络有助于使这些系统具有鲁棒性，从而使诸如释义之类的效果，但仍主要捕获浅表文本模式。要回答有关文本中事件之间因果关系的更深入的问题，系统可能需要结合几个信息，抽象无关紧要的细节，并结合了以前的世界知识以得出答案。该项目旨在开发可以应对这些挑战的系统：这些系统明确地对文本进行了建模推理，并利用神经网络以细微的方式进行推理的力量。这种推理是通过“手持”监督明确教授到系统的，这鼓励系统模仿人类如何解决问题，并帮助他们更好地推广到新的问题实例。这种与人类所做的事情的一致性也有助于揭示系统的决策过程；它提供了一种对其行为的解释形式，以便可以根据所需的标准（例如公平性）对其进行评估。该提案的技术创新集中在两个方面：设计潜在变量模型并在模型培训期间利用新型的手持式监督。这些技术是在需要复杂推理的三个具有挑战性的问题的背景下探索的：（1）解决数学单词问题；（2）使用世界知识解决核心；（3）从文件中回答问题。对于每个问题，提出了新的模型围绕答案的离散推导，该模型借鉴了最新工具，例如基于注意力的重复神经网络，以捕获推理过程的更大背景。模型决策的离散性为纳入辅助监督提供了锚点，这在完全端到端的神经模型中很难做到。手持监督的性质取决于任务，是偶然的监督，启发式识别的推导和针对人类注释的结合。每个解决的问题测试方法的不同方面，例如处理复杂的推导并纳入世界知识，以及这些问题产生具体评估框架，以了解拟议技术的功效。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子优点和广泛的影响来评估CRERIA的评估。

项目成果

期刊论文数量（14）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Understanding Dataset Design Choices for Multi-hop Reasoning

DOI：
10.18653/v1/n19-1405
发表时间：
2019
期刊：
影响因子：
0
作者：
Jifan Chen;Greg Durrett
通讯作者：
Jifan Chen;Greg Durrett

Generating Literal and Implied Subquestions to Fact-check Complex Claims

生成字面和隐含的子问题来事实检查复杂的声明

DOI：
发表时间：
2022
期刊：
Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP
影响因子：
0
作者：
Chen, Jifan;Sriram, Aniruddh;Choi, Eunsol;Durrett, Greg
通讯作者：
Durrett, Greg

The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning