CRCNS: Information Flow in the Brain During Language and Meaning Comprehension

描述（由申请人提供）：在过去的十年里，我们从功能磁共振成像研究中了解到很多关于人类大脑用来表示单词和概念的意义的神经激活的空间模式。人们对这种神经活动的时间演变知之甚少，包括大脑在几百毫秒内理解一个单词所使用的时间相关的子过程，或者当单词一个接一个到达时，大脑用来构建和编码整个句子的意义的更复杂的过程。我们建议研究在理解单个单词、短语、句子和故事时观察到的详细的时空神经活动，并建立计算模型。这项拟议的研究将专门针对以下问题：“在孤立地理解一个单词所需的时间内，神经活动在何时何地以及由大脑中的哪些子过程编码了哪些信息？”“当一个新感知的单词首先激活感觉皮层，然后导致神经激活编码单词含义时，编码的信息流是什么？”“大脑如何将新遇到的单词整合到句子或短语的早期单词的上下文中，以组成多词短语或句子的意义表示？”以及“与孤立地处理相同的单词或作为非结构化的集合（如{kick， Joe, ball}）相比，语义期望和要求以及句法句子结构如何改变单词的处理？”为了研究这些问题，我们将(1)设计新的实验方案来探测单词和句子处理过程中神经信号中编码的信息流；(2)使用功能磁共振成像（fMRI）收集新的脑图像数据，以达到几毫米的空间分辨率，MEG达到几毫秒的时间分辨率；(3)开发和应用新的机器学习方法来构建集成和预测这些组合实验数据的计算模型。我们的目标是开发一个越来越精确的计算模型来研究大脑是如何理解单词、短语和句子的——这个模型可以对观察到的神经活动做出可测试的预测，这些神经活动是对新的语言刺激做出反应的。智力优势：这项合作研究将先进的机器学习算法与MEG和fMRI脑成像的新实验协议结合在一起，以推进我们对人类大脑两个基本开放问题的理解：大脑如何表示意义，以及从感知到的语言刺激中，哪些神经认知过程一点一点地构建意义？更广泛的影响：如果成功，这项研究将影响广泛的社区，包括(1)认知神经科学和计算语言学，提供对大脑语言处理的更好理解；(2)机器学习，通过推动时间序列和潜在变量分析的新方法的发展，整合多个数据集，并结合多样化

DESCRIPTION (provided by applicant): Over the past ten years a good deal has been learned from fMRI studies about the spatial patterns of neural activation used by the human brain to represent meanings of words and concepts. Much less is understood about the time evolution of this neural activity, including the temporally interrelated sub-processes the brain employs during the hundreds of milliseconds it takes to comprehend a single word, or the more complex processes it uses to construct and encode meaning of entire sentences as the words arrive one by one. We propose research to study, and to build computational models of, the detailed spatial and temporal neural activity observed during the comprehension of single words, phrases, sentences, and stories. This proposed research will specifically target the following questions: "What information is encoded by neural activity where and when, and by which subprocesses in the brain, during the time it takes to comprehend a single word in isolation?" "What is the flow of information encoded when a newly sensed word first activates sensory cortex, then later results in neural activation encoding the word meaning?" "How does the brain integrate a newly encountered word in the context of earlier words in the sentence or phrase, to compose the meaning representation of the multi-word phrase or sentence?" and "How do semantic expectations and demands, together with syntactic sentence structure alter the processing of words, compared to processing the same words in isolation, or as an unstructured set such as {kick, Joe, ball}?" To study these questions we will (1) devise novel experimental protocols to probe the flow of information encoded in neural signals during word and sentence processing, (2) collect new brain image data using both fMRI to achieve spatial resolution of a few millimeters, and MEG to achieve temporal resolution of a few milliseconds, (3) develop and apply novel machine learning approaches to build computational models that integrate and that predict this combined experimental data. Our goal is to develop an increasingly accurate computational model of how the brain comprehends words, phrases and sentences - a model that makes testable predictions about the neural activity observed in response to novel language stimuli. Intellectual Merit: This collaborative research brings together advanced machine learning algorithms with novel experimental protocols for MEG and fMRI brain imaging to advance our understanding of two fundamental open questions about the human brain: how does the brain represent meaning, and what neuro-cognitive processes construct that meaning piece-by-piece from perceived language stimuli? Broader Impacts: If successful, this research will impact a broad range of communities, including (1) cognitive neuroscience and computational linguistics, providing improved understanding of language processing in the brain, (2) machine learning, by driving the development of new methods for time series and latent variable analysis, integrating multiple data sets, and incorporating diverse background knowledge as priors, (3) clinical studies of brain pathologies, especially those related to language processing, and informing treatment strategies for developmental and acquired language disorders (4) education of graduates, undergraduates and the general public, through dissemination of technical articles, teaching materials, and news about our work in the public press.