CRCNS: Discovering Computational Principles of Language Processing in the Brain

人类语言的一个重要特征是我们能够理解多词序列， 意义大于各部分的总和该提案的PI最近的工作（Toneva和Wehbe， 2019; Jain和Huth，2018）和其他人（Schrimpf等人，2020 a; Caucheteux & King，2020）表明， 多词序列的皮层表示可以比以前更准确地建模， 使用神经网络语言模型，这是一种机器学习技术，它彻底改变了自然语言， 语言处理（NLP）领域（Devlin等人，2019;拉德福等人，2019年）。然而，在目前 这些模型必须首先在单独的NLP任务上进行训练，然后才能用于对大脑进行建模， 创造了一个猜测和检查的循环，不能保证收敛于实际计算， 人类表演。在这里，我们建议通过直接训练神经网络模型来打破这个循环， 估计大脑用来组合联合收割机单词的功能。为了能够最佳地预测功能磁共振成像和脑磁图 响应，这些模型将需要捕捉的组成原则，支配哪些词的大脑 以及信息是如何在单词之间组合的。这些模型将有助于揭示特定的 大脑中语言处理的基础计算，使神经语言学的计算测试成为可能。 理论，并启发或直接改进NLP中使用的模型。 然而，实现这些目标需要克服一个主要障碍。训练神经网络- 工作语言模型通常需要比现有神经成像多几个数量级的数据 数据集。为了解决这个问题，这个拟议项目的一个中心目标是收集一个非常大的功能磁共振成像， MEG数据集包含大约100万个自然语言刺激词。我们计划用独一无二的 数据集和计算建模框架，以解决三个科学目标。 目标1：创建大脑活动预测基准，以促进神经科学和NLP之间的互动。 目标2：使用数据驱动模型来测试现有的神经语言学理论，并开发新的解释。 大脑中单词组成的基础计算。目标3：利用不同大脑中的信息 帮助解决计算定义的语言任务的区域。 成功完成拟议的工作将提供对语言处理的机械见解， 用一个计算架构来追踪大脑区域之间的信息流，并描述它们所执行的任务， 表演。除了其基本的认知神经科学的影响，我们预计这项工作将使更好地 了解语言障碍，并帮助确定有针对性的治疗。 相关性（参见说明）： 通过收集、分析和传播大规模神经成像数据集， 参与者听自然的，叙述性的演讲，这项建议旨在提高我们对 语言系统的正常功能。具体而言，这项工作旨在改善和验证 人类大脑中语音语言处理的计算模型。

A critical characteristic of human language is our ability to understand multi-word sequences whose meaning is greater than the sum of their parts. Recent work from the PIs of this proposal (Toneva and Wehbe, 2019; Jain and Huth, 2018) and others (Schrimpf et al., 2020a; Caucheteux & King, 2020) has shown that cortical representations of multi-word sequences can be modeled much more accurately than before by using neural network language models, a machine learning technique that has revolutionized the natural language processing (NLP) field (Devlin et al., 2019; Radford et al., 2019). However, under the current paradigm these models must first be trained on separate NLP tasks and only then used to model the brain, creating a guess-and-check cycle that is not guaranteed to converge on the actual computations that humans perform. Here we propose to break this cycle by directly training neural network models to estimate the functions that the brain uses to combine words. To be able to optimally predict fMRI and MEG responses, these models will need to capture the composition principles governing which words the brain attends to, and how information is combined across words. These models will help uncover specific computations underlying language processing in the brain, enable computational testing of neurolinguistic theories, and inspire or directly improve models used in NLP. Accomplishing these goals, however, will require overcoming one major obstacle. Training neural net- work language models typically requires orders of magnitude more data than existing neuroimaging datasets. To address this issue, one central goal of this proposed project is to collect a very large fMRI and MEG dataset comprising roughly one million words of natural language stimuli. We plan to use the unique dataset and computational modeling framework to address three scientific aims. Aim 1: Create brain activity prediction benchmarks to foster interaction between neuroscience and NLP. Aim 2: Use data-driven models to test existing neurolinguistic theories & develop new accounts of the computations underlying word composition in the brain. Aim 3: Leverage information in different brain areas to help solve computationally defined language tasks. Successful completion of the proposed work will provide mechanistic insight into language processing, with a computational architecture tracing information flow among brain areas and describing the tasks they perform. Beyond its basic cognitive neuroscience implications, we expect this work will enable better understanding of language impairments and help identify targeted therapies. RELEVANCE (See instructions): Through collecting, analyzing, and disseminating large-scale neuroimaging datasets collected while participants listen to natural, narrative speech, this proposal aims to improve our understanding of the normal function of the language system. Specifically, this work seeks to improve and validate computational models of speech language processing in the human brain.