Joint Text Understanding and Coreference Resolution: Augmenting Recurrent Neural Networks with Entity-Centric Discourse Memory

联合文本理解和共指解析：用以实体为中心的话语记忆增强循环神经网络

• 批准号: 1895643
• 金额: --
• 依托单位: University of Oxford
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2017
• 资助国家: 英国
• 起止时间: 2017 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-1895643
• 关键词: Joint; Text; Understanding; Coreference; Resolution

This research project falls within the natural language processing, artificial intelligence technologies, and human-computer interaction research areas of EPSRC primarily within the ICT theme.Despite recent advances in machine learning and natural language processing, accurate text comprehension has remained a challenging problem. Prior works often used recurrent neural networks that embed the question, passage, and each answer candidate into real-valued vectors, optionally with an additional attention mechanism.Such approaches have two drawbacks. First, the finite capacity of the neural network hidden states means that the model has to compress all the relevant information within the network capacity. For instance, the model needs to make the connection that mentions of "Hillary Clinton" and "Clinton" refer to the same person (i.e. coreference resolution task), along with the relation between different entities. Second, the resulting vector embedding is often not easily interpretable to humans.This research area is an important one, since any meaningful interaction between humans and computers through text or speech requires the ability to properly understand and reason about meaning and context of natural language. Designing intelligent systems that can recognize entities and the relations between them as part of text comprehension would give rise to more accurate and interpretable text comprehension systems.The aim of this research project is to design statistical models and algorithms that can jointly perform coreference resolution and text comprehension, using an entity-centric memory that stores long-range discourse features. Recent work have demonstrated that reasoning about entities and references is beneficial for language modeling and text comprehension, although the proposed approach learns such coreference information automatically and jointly with the text comprehension task. A second goal of this approach is to improve model interpretability by associating each information with the relevant entity, and explicitly making predictions about coreference links and entity relations.Our proposed approach similarly leverages the strength of recurrent neural networks and attention mechanisms, but also augments the model with a "discourse memory" that keeps track of each entity and its various mentions within the text. This approach has the benefit of integrating all pertinent information regarding an entity into the same file in a computationally efficient manner, even though each information is mentioned separately and may occur distantly in the passage. While some prior work have similarly investigated an entity-centric approach to text comprehension with discourse-level features, the novelty of our approach additionally lies in performing the coreference resolution jointly with the text comprehension task, which remains a challenging task.

本研究项目福尔斯属于EPSRC的自然语言处理、人工智能技术和人机交互研究领域，主要是ICT主题。尽管机器学习和自然语言处理最近取得了进展，但准确的文本理解仍然是一个具有挑战性的问题。以前的工作通常使用递归神经网络，将问题，段落和每个答案候选者嵌入到实值向量中，可选地具有额外的注意力机制。首先，神经网络隐藏状态的有限容量意味着模型必须压缩网络容量内的所有相关信息。例如，该模型需要建立“Hillary Clinton”和“Clinton”提到的是同一个人的联系（即共指消解任务），沿着不同实体之间的关系。第二，所得到的向量嵌入通常不容易被人类解释，这是一个重要的研究领域，因为人类和计算机之间通过文本或语音进行的任何有意义的交互都需要正确理解和推理自然语言的含义和上下文的能力。设计能够识别实体及其之间关系的智能系统作为文本理解的一部分，将产生更准确和可解释的文本理解系统。该研究项目的目的是设计能够联合执行共指消解和文本理解的统计模型和算法，使用存储远程话语特征的以实体为中心的记忆。最近的工作表明，推理的实体和参考是有益的语言建模和文本理解，虽然所提出的方法自动学习这种共指信息，并与文本理解任务。这种方法的第二个目标是通过将每个信息与相关实体相关联来提高模型的可解释性，并明确地对共指链接和实体关系进行预测。我们提出的方法同样利用了递归神经网络和注意力机制的优势，但也增强了模型的“话语记忆”，它可以跟踪文本中的每个实体及其各种提及。这种方法的好处是以计算高效的方式将关于实体的所有相关信息集成到同一文件中，即使每个信息被单独提及并且可能在段落中遥远地出现。虽然之前的一些工作也类似地研究了以实体为中心的具有语篇级别特征的文本理解方法，但我们方法的新奇还在于将共指消解与文本理解任务联合执行，这仍然是一项具有挑战性的任务。