EAGER: Collaborative Research: World Modeling for Natural Language Understanding

人工智能(AI)的一个关键目标是建立能够像人类一样阅读和理解语言的系统。这一能力是一系列技术的基础，包括问题回答、机器翻译和对话系统。虽然已经取得了进展，但人工智能系统目前缺乏人类语言理解的稳健性和灵活性-典型的系统利用浅模式匹配策略来执行任务，因此只在它们为之构建的特定任务中有效，即使在这些环境中也很容易失败。这个项目通过提高系统构建文本中描述的“世界”的丰富表示的能力来解决这些问题：谁是参与其中的实体，它们的属性和关系是什么？正在发生什么活动，谁参加了这些活动，为什么会发生这些活动？这些系统的世界概念的设计使用了认知科学家和心理学家认为是人类语言理解基础的概念。这项工作的预期好处是开发出能够灵活而稳健地使用语言的人工智能系统，因为像人类一样，这些系统将基于语言传达的核心信息执行任务，而不是表面的模式匹配。除了改进系统，该项目还将在人工智能社区与认知科学家、心理学家和语言学家之间建立桥梁-该项目的建模框架提供了一条途径，通过该途径，来自认知科学的见解可以转化为模型实现，这可以用于改进人工智能系统和测试认知假设。这个探索性的急切项目提高了系统自动构建所分析文本的基础世界的能力，并设计了有针对性的探测任务，以实现对系统捕获此信息的程度的细粒度评估。该建模框架使用记忆增强的神经网络，利用外部记忆组件来表示世界。该项目不是明确的注释，而是实现世界组件本身的认知启发设计和诱导偏见，以鼓励特定组件捕捉预期的内容。学习是通过自我监督的目标和对大数据叙事的辅助监督来进行的。系统评价包括标准阅读理解问答任务和新奇探究任务的发展。受控探测任务的使用关键地借鉴了认知神经科学和心理语言学中使用的方法论方法，将这些科学方法应用于解释人工系统。这些探测任务允许对单个世界组件进行有针对性的分析，并为模型改进提供指导。该项目的方法通过探测任务在模型设计和目标测试之间迭代，使用后者的结果来指导前者。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

A key goal of artificial intelligence (AI) is to build systems that can read and understand language as humans do. This capability underlies a broad range of technologies, including question answering, machine translation, and dialogue systems. While progress has been made, AI systems currently lack the robustness and flexibility of human language understanding---typical systems leverage shallow pattern-matching strategies to perform tasks, and as a result are only effective at the specific tasks they are built for, and fail easily even within those settings. This project addresses these issues by improving the ability of systems to construct rich representations of the "world" described in text: Who are the entities involved, and what are their attributes and relationships? What events are taking place, who is participating in those events, and why are they occurring? The design of the systems' notion of a world uses concepts like these that have been identified by cognitive scientists and psychologists as fundamental in human language understanding. The expected benefit of this work is the development of AI systems that can use language flexibly and robustly because, like humans, these systems will perform tasks based on the core information conveyed in language, rather than superficial pattern-matching. In addition to improving systems, this project will have the benefit of building bridges between the AI community and cognitive scientists, psychologists, and linguists---the project's modeling framework provides a pathway through which insights from cognitive science can be translated to model implementation, which can be utilized both for improvement of AI systems and for testing of cognitive hypotheses. This exploratory EAGER project improves the capacity of systems to automatically construct the world underlying the text being analyzed, and designs targeted probing tasks to enable fine-grained assessment of the extent to which systems have captured this information. The modeling framework uses memory-augmented neural networks, leveraging the external memory components to represent worlds. Rather than explicit annotation, the project implements cognitively-inspired design of both world components themselves and inductive bias for encouraging particular components to capture what is intended. Learning is carried out via self-supervised objectives and auxiliary supervision on large datasets of narratives. System evaluation consists of both standard reading comprehension question answering tasks and the development of novel probing tasks. The use of controlled probing tasks draws critically from methodological approaches used in cognitive neuroscience and psycholinguistics, applying these scientific methods for interpretation of artificial systems. These probing tasks allow for targeted analysis of individual world components and provide guidance for model improvement. The methodology of the project iterates between model design and targeted testing via probing tasks, using the results of the latter to guide the former.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.