CRCNS: Discovering Computational Principles of Language Processing in the Brain

人类语言的一个关键特征是我们理解多个单词序列的能力，这些序列 意义大于它们各部分之和。该提案的PI最近的工作(Toneva和Wehbe， 2019年；Jain和Huth，2018)和其他人(Schrimpf等人，2020a；Caucheteux&King，2020)已经表明 多个单词序列的皮层表示可以比以前更准确地建模，通过 使用神经网络语言模型，这是一种机器学习技术，它彻底改变了自然 语言处理(NLP)领域(Devlin等人，2019年；Radford等人，2019年)。然而，在当前的 这些模型必须首先在单独的NLP任务上进行训练，然后才能用于对大脑进行建模， 创建一个猜测和检查循环，不能保证在实际计算上收敛 人类的表现。在这里，我们建议通过直接训练神经网络模型来打破这种循环 估计大脑用来组合单词的功能。能够最佳地预测功能磁共振成像和脑磁图 这些模型需要捕捉控制大脑中哪些单词的组成原则 关注信息是如何跨词语组合的。这些模型将有助于揭示特定的 大脑中潜在的语言处理的计算，使神经语言学的计算测试成为可能 理论，并启发或直接改进NLP中使用的模型。 然而，要实现这些目标，需要克服一个主要障碍。训练神经网络- 工作语言模型通常需要比现有神经成像多几个数量级的数据 数据集。为了解决这个问题，这个拟议项目的一个中心目标是收集非常大的fMRI和 包括大约一百万个自然语言刺激词的MEG数据集。我们计划使用独特的 数据集和计算建模框架，以解决三个科学目标。 目标1：创建脑活动预测基准，以促进神经科学和NLP之间的互动。 目标2：使用数据驱动模型来测试现有的神经语言学理论，并开发新的帐户 大脑中潜在的单词组成的计算。目标3：利用不同大脑中的信息 帮助解决计算定义的语言任务的领域。 拟议工作的成功完成将提供对语言处理的机械性洞察， 通过计算架构跟踪大脑区域之间的信息流并描述它们的任务 表演。除了其基本的认知神经科学含义外，我们预计这项工作将使 了解语言障碍并帮助确定有针对性的治疗方法。 相关性(请参阅说明)： 通过收集、分析和传播收集的大规模神经成像数据 与会者聆听自然、叙事性的演讲，这一建议旨在提高我们对 语言系统的正常功能。具体地说，这项工作旨在改进和验证 人脑中语音语言处理的计算模型。

A critical characteristic of human language is our ability to understand multi-word sequences whose meaning is greater than the sum of their parts. Recent work from the PIs of this proposal (Toneva and Wehbe, 2019; Jain and Huth, 2018) and others (Schrimpf et al., 2020a; Caucheteux & King, 2020) has shown that cortical representations of multi-word sequences can be modeled much more accurately than before by using neural network language models, a machine learning technique that has revolutionized the natural language processing (NLP) field (Devlin et al., 2019; Radford et al., 2019). However, under the current paradigm these models must first be trained on separate NLP tasks and only then used to model the brain, creating a guess-and-check cycle that is not guaranteed to converge on the actual computations that humans perform. Here we propose to break this cycle by directly training neural network models to estimate the functions that the brain uses to combine words. To be able to optimally predict fMRI and MEG responses, these models will need to capture the composition principles governing which words the brain attends to, and how information is combined across words. These models will help uncover specific computations underlying language processing in the brain, enable computational testing of neurolinguistic theories, and inspire or directly improve models used in NLP. Accomplishing these goals, however, will require overcoming one major obstacle. Training neural net- work language models typically requires orders of magnitude more data than existing neuroimaging datasets. To address this issue, one central goal of this proposed project is to collect a very large fMRI and MEG dataset comprising roughly one million words of natural language stimuli. We plan to use the unique dataset and computational modeling framework to address three scientific aims. Aim 1: Create brain activity prediction benchmarks to foster interaction between neuroscience and NLP. Aim 2: Use data-driven models to test existing neurolinguistic theories & develop new accounts of the computations underlying word composition in the brain. Aim 3: Leverage information in different brain areas to help solve computationally defined language tasks. Successful completion of the proposed work will provide mechanistic insight into language processing, with a computational architecture tracing information flow among brain areas and describing the tasks they perform. Beyond its basic cognitive neuroscience implications, we expect this work will enable better understanding of language impairments and help identify targeted therapies. RELEVANCE (See instructions): Through collecting, analyzing, and disseminating large-scale neuroimaging datasets collected while participants listen to natural, narrative speech, this proposal aims to improve our understanding of the normal function of the language system. Specifically, this work seeks to improve and validate computational models of speech language processing in the human brain.