CRII: RI: Opening the black box of neural natural language processing models using machine-behavioral methods

CRII：RI：使用机器行为方法打开神经自然语言处理模型的黑匣子

• 批准号: 1947307
• 负责人: Richard Futrell
• 金额: $ 17.34万
• 依托单位: University of California-Irvine
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-07-01 至 2023-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1947307&HistoricalAwards=false
• 关键词: CRII; RI; Opening; black; box

In our daily lives, we are increasingly interacting with computer applications using natural language. The goal of this project is to understand and improve the neural network technology that underlies these applications. A neural network is an artificial intelligence system that is trained to behave in a certain way by showing it many examples of how it should behave. While neural networks are the best way currently known for building natural language applications, they have a major drawback: Once they appear to have learned how to do some natural language task, we don’t know exactly what they have learned, and consequently we can’t be certain how they will act in all circumstances. In order to improve this technology to be more robust, and also to make it more comprehensible and controllable, we need to develop ways of determining exactly what patterns a neural network has learned once it has been trained. In this project, methods from experimental psychology are adapted and used to reveal the inner workings of neural network systems that have been trained to perform natural language tasks. This research focuses on determining how well neural networks that are trained to do natural language processing (NLP) tasks have learned representations of the syntactic structure of natural language: the grammatical relationships among words in sentences. It does so by adopting the recently proposed “machine behavior” approach in which neural networks are studied by subjecting them to behavioral tests. The project as a whole has three major components. The first is to develop and train a large reference set of neural network NLP systems that vary in their architecture; all subsequent experiments are to be carried out on all of these models, so that they can be ranked in terms of the quality of their representations of syntactic structure. The second is to carry out a number of behavioral tests, by constructing carefully crafted sentences designed to elicit certain responses from the neural networks. The third is to analyze the actual information inside of the neural networks using structural probing, a newly developed technique, in order to detect representations of syntactic structure directly. The results of this research can be useful for guiding future NLP research in the development of more comprehensible neural network systems for natural language applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在我们的日常生活中，我们越来越多地使用自然语言与计算机应用程序进行交互。该项目的目标是了解和改进这些应用程序的神经网络技术。神经网络是一种人工智能系统，它通过向它展示它应该如何表现的许多例子来训练它以某种方式表现。虽然神经网络是目前已知的构建自然语言应用程序的最佳方法，但它们有一个主要缺点：一旦它们似乎学会了如何完成某些自然语言任务，我们就不知道它们到底学到了什么，因此我们无法确定它们在所有情况下将如何行动。为了改进这项技术，使其更强大，也使其更易于理解和控制，我们需要开发出一种方法，来确定神经网络在训练后究竟学到了什么样的模式。在这个项目中，来自实验心理学的方法被改编并用于揭示经过训练以执行自然语言任务的神经网络系统的内部工作原理。这项研究的重点是确定被训练来执行自然语言处理（NLP）任务的神经网络学习自然语言句法结构的表现：句子中单词之间的语法关系。它通过采用最近提出的“机器行为”方法来做到这一点，在这种方法中，神经网络通过对它们进行行为测试来研究。整个项目有三个主要组成部分。第一个是开发和训练一个大型的参考神经网络NLP系统，这些系统的架构各不相同;所有后续的实验都将在所有这些模型上进行，这样就可以根据句法结构表示的质量对它们进行排名。第二种方法是进行一些行为测试，通过构建精心设计的句子来引发神经网络的某些反应。第三种是使用结构探测（一种新开发的技术）分析神经网络内部的实际信息，以便直接检测句法结构的表示。这项研究的结果可以用于指导未来的NLP研究，开发更容易理解的神经网络系统，用于自然语言应用。该奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

项目成果

Explaining patterns of fusion in morphological paradigms using the memory-surprisal tradeoff

• 发表时间: 2022
• 作者: Neil Rathi;Michael Hahn;Richard Futrell

When classifying grammatical role, BERT doesn’t care about word order... except when it matters

在对语法角色进行分类时，BERT 并不关心词序……除非它很重要

• DOI: 10.18653/v1/2022.acl-short.71
• 发表时间: 2022
• 期刊: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers
• 作者: Papadimitriou, Isabel;Futrell, Richard;Mahowald, Kyle
• 通讯作者: Mahowald, Kyle

An Information-Theoretic Characterization of Morphological Fusion

形态融合的信息论表征

• DOI: 10.18653/v1/2021.emnlp-main.793
• 发表时间: 2021
• 期刊: An Information-Theoretic Characterization of Morphological Fusion
• 作者: Rathi, Neil;Hahn, Michael;Futrell, Richard
• 通讯作者: Futrell, Richard

Estimating word co-occurrence probabilities from pretrained static embeddings using a log-bilinear model

使用对数双线性模型从预训练的静态嵌入中估计单词共现概率

• DOI: 10.18653/v1/2022.cmcl-1.6
• 发表时间: 2022
• 期刊: Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics
• 作者: Futrell, Richard

Sensitivity as a Complexity Measure for Sequence Classification Tasks

• DOI: 10.1162/tacl_a_00403
• 发表时间: 2021-04
• 期刊: Transactions of the Association for Computational Linguistics
• 影响因子: 10.9
• 作者: Michael Hahn;Dan Jurafsky;Richard Futrell