EAGER: Collaborative Research: World Modeling for Natural Language Understanding

人工智能（AI）的一个关键目标是构建能够像人类一样阅读和理解语言的系统。这种能力是一系列技术的基础，包括问答、机器翻译和对话系统。虽然已经取得了进展，但人工智能系统目前缺乏人类语言理解的稳健性和灵活性——典型的系统利用肤浅的模式匹配策略来执行任务，因此只能在特定的任务中有效，即使在这些设置中也很容易失败。该项目通过提高系统构建文本中描述的“世界”的丰富表示的能力来解决这些问题：涉及的实体是谁，它们的属性和关系是什么？正在发生什么事件，谁参与了这些事件，为什么会发生这些事件？系统对世界概念的设计使用了认知科学家和心理学家认为的这些概念，这些概念是人类语言理解的基础。这项工作的预期好处是开发能够灵活而稳健地使用语言的人工智能系统，因为像人类一样，这些系统将基于语言传达的核心信息执行任务，而不是表面的模式匹配。除了改进系统之外，该项目还将有助于在人工智能社区与认知科学家、心理学家和语言学家之间建立桥梁——该项目的建模框架提供了一条途径，通过该途径，认知科学的见解可以转化为模型实施，这既可以用于改进人工智能系统，也可以用于测试认知假设。这个探索性EAGER项目提高了系统自动构建被分析文本的世界的能力，并设计了有针对性的探测任务，以便对系统捕获该信息的程度进行细粒度评估。建模框架使用内存增强神经网络，利用外部内存组件来表示世界。与显式注释不同，该项目实现了世界组件本身的认知启发设计和归纳偏差，以鼓励特定组件捕获预期的内容。学习是通过自我监督目标和对大型叙事数据集的辅助监督来进行的。系统评价包括标准的阅读理解问题回答任务和开发新的探究任务。控制探测任务的使用从认知神经科学和心理语言学中使用的方法论方法中吸取了批判性的教训，将这些科学方法应用于人工系统的解释。这些探索任务允许对单个世界组件进行有针对性的分析，并为模型改进提供指导。项目的方法在模型设计和通过探测任务的目标测试之间迭代，使用后者的结果来指导前者。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

A key goal of artificial intelligence (AI) is to build systems that can read and understand language as humans do. This capability underlies a broad range of technologies, including question answering, machine translation, and dialogue systems. While progress has been made, AI systems currently lack the robustness and flexibility of human language understanding---typical systems leverage shallow pattern-matching strategies to perform tasks, and as a result are only effective at the specific tasks they are built for, and fail easily even within those settings. This project addresses these issues by improving the ability of systems to construct rich representations of the "world" described in text: Who are the entities involved, and what are their attributes and relationships? What events are taking place, who is participating in those events, and why are they occurring? The design of the systems' notion of a world uses concepts like these that have been identified by cognitive scientists and psychologists as fundamental in human language understanding. The expected benefit of this work is the development of AI systems that can use language flexibly and robustly because, like humans, these systems will perform tasks based on the core information conveyed in language, rather than superficial pattern-matching. In addition to improving systems, this project will have the benefit of building bridges between the AI community and cognitive scientists, psychologists, and linguists---the project's modeling framework provides a pathway through which insights from cognitive science can be translated to model implementation, which can be utilized both for improvement of AI systems and for testing of cognitive hypotheses. This exploratory EAGER project improves the capacity of systems to automatically construct the world underlying the text being analyzed, and designs targeted probing tasks to enable fine-grained assessment of the extent to which systems have captured this information. The modeling framework uses memory-augmented neural networks, leveraging the external memory components to represent worlds. Rather than explicit annotation, the project implements cognitively-inspired design of both world components themselves and inductive bias for encouraging particular components to capture what is intended. Learning is carried out via self-supervised objectives and auxiliary supervision on large datasets of narratives. System evaluation consists of both standard reading comprehension question answering tasks and the development of novel probing tasks. The use of controlled probing tasks draws critically from methodological approaches used in cognitive neuroscience and psycholinguistics, applying these scientific methods for interpretation of artificial systems. These probing tasks allow for targeted analysis of individual world components and provide guidance for model improvement. The methodology of the project iterates between model design and targeted testing via probing tasks, using the results of the latter to guide the former.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.