CRCNS: Information Flow in the Brain During Language and Meaning Comprehension

CRCNS：语言和意义理解过程中大脑中的信息流

• 批准号: 8532012
• 负责人: MARCEL Adam JUST
• 金额: $ 22.7万
• 依托单位: CARNEGIE-MELLON UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-08-16 至 2017-05-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/8532012
• 关键词: Acquired Language Disorders; Algorithms; Automobile Driving; Brain; Brain Pathology; Brain imaging; Clinical Research; Cognitive; Communities; Complex; Comprehension; Computer Simulation; Data; Data Set; Development; Evolution; Functional Magnetic Resonance Imaging; General Population; Goals; Graduate Education; Human; Knowledge; Language; Learning; Linguistics; Machine Learning; Methods; Modeling; Process; Protocols documentation; Research; Resolution; Semantics; Series; Signal Transduction; Stimulus; Structure; Teaching Materials; Time; Word Processing; Work; cognitive neuroscience; expectation; improved; language processing; millimeter; millisecond; neural patterning; news; novel; phrases; relating to nervous system; response; sensory cortex; syntax; treatment strategy

DESCRIPTION (provided by applicant): Over the past ten years a good deal has been learned from fMRI studies about the spatial patterns of neural activation used by the human brain to represent meanings of words and concepts. Much less is understood about the time evolution of this neural activity, including the temporally interrelated sub-processes the brain employs during the hundreds of milliseconds it takes to comprehend a single word, or the more complex processes it uses to construct and encode meaning of entire sentences as the words arrive one by one. We propose research to study, and to build computational models of, the detailed spatial and temporal neural activity observed during the comprehension of single words, phrases, sentences, and stories. This proposed research will specifically target the following questions: "What information is encoded by neural activity where and when, and by which subprocesses in the brain, during the time it takes to comprehend a single word in isolation?" "What is the flow of information encoded when a newly sensed word first activates sensory cortex, then later results in neural activation encoding the word meaning?" "How does the brain integrate a newly encountered word in the context of earlier words in the sentence or phrase, to compose the meaning representation of the multi-word phrase or sentence?" and "How do semantic expectations and demands, together with syntactic sentence structure alter the processing of words, compared to processing the same words in isolation, or as an unstructured set such as {kick, Joe, ball}?" To study these questions we will (1) devise novel experimental protocols to probe the flow of information encoded in neural signals during word and sentence processing, (2) collect new brain image data using both fMRI to achieve spatial resolution of a few millimeters, and MEG to achieve temporal resolution of a few milliseconds, (3) develop and apply novel machine learning approaches to build computational models that integrate and that predict this combined experimental data. Our goal is to develop an increasingly accurate computational model of how the brain comprehends words, phrases and sentences - a model that makes testable predictions about the neural activity observed in response to novel language stimuli. Intellectual Merit: This collaborative research brings together advanced machine learning algorithms with novel experimental protocols for MEG and fMRI brain imaging to advance our understanding of two fundamental open questions about the human brain: how does the brain represent meaning, and what neuro-cognitive processes construct that meaning piece-by-piece from perceived language stimuli? Broader Impacts: If successful, this research will impact a broad range of communities, including (1) cognitive neuroscience and computational linguistics, providing improved understanding of language processing in the brain, (2) machine learning, by driving the development of new methods for time series and latent variable analysis, integrating multiple data sets, and incorporating diverse background knowledge as priors, (3) clinical studies of brain pathologies, especially those related to language processing, and informing treatment strategies for developmental and acquired language disorders (4) education of graduates, undergraduates and the general public, through dissemination of technical articles, teaching materials, and news about our work in the public press.

描述（由申请人提供）：在过去的十年中，人们从功能磁共振成像研究中学到了大量关于人脑用来表示单词和概念含义的神经激活空间模式的知识。人们对这种神经活动的时间演化了解甚少，包括大脑在理解单个单词所需的数百毫秒内使用的时间上相互关联的子过程，或者当单词一个接一个地到达时大脑用来构建和编码整个句子的含义的更复杂的过程。我们提出研究并建立计算模型，研究在理解单个单词、短语、句子和故事过程中观察到的详细空间和时间神经活动。这项拟议的研究将专门针对以下问题：“在单独理解单个单词所需的时间内，神经活动在何时何地以及通过大脑中的哪些子过程编码了哪些信息？” “当新感知到的单词首先激活感觉皮层，然后导致编码单词含义的神经激活时，编码的信息流是什么？” “大脑如何将新遇到的单词整合到句子或短语中较早单词的上下文中，以构成多单词短语或句子的含义表示？”和“与单独处理相同的单词或作为非结构化集合（例如{kick，Joe，ball}）相比，语义期望和要求以及句法句子结构如何改变单词的处理？”为了研究这些问题，我们将 (1) 设计新颖的实验方案来探测单词和句子处理过程中神经信号中编码的信息流，(2) 使用功能磁共振成像 (fMRI) 来实现几毫米的空间分辨率，并使用 MEG 来实现几毫秒的时间分辨率，收集新的大脑图像数据，(3) 开发和应用新颖的机器学习方法来构建集成并预测这些组合实验数据的计算模型。我们的目标是开发一个越来越准确的大脑如何理解单词、短语和句子的计算模型，该模型可以对响应新语言刺激所观察到的神经活动做出可测试的预测。智力优势：这项合作研究将先进的机器学习算法与脑磁图和功能磁共振成像大脑成像的新颖实验协议结合在一起，以增进我们对有关人脑的两个基本开放问题的理解：大脑如何表示意义，以及什么神经认知过程从感知到的语言刺激中逐段构建该意义？更广泛的影响：如果成功，这项研究将影响广泛的社区，包括（1）认知神经科学和计算语言学，提高对大脑语言处理的理解，（2）机器学习，推动时间序列和潜在变量分析新方法的开发，整合多个数据集，并结合不同的数据集 背景知识作为先验知识，(3) 脑病理学的临床研究，特别是与语言处理相关的脑病理学，并为发展性和后天性语言障碍的治疗策略提供信息 (4) 通过在公共媒体上传播有关我们工作的技术文章、教材和新闻，对研究生、本科生和公众进行教育。