CRCNS: Information Flow in the Brain During Language and Meaning Comprehension

描述(申请人提供)：在过去的十年里，从fMRI研究中学到了许多关于人脑用来表示单词和概念含义的神经激活的空间模式。关于这一神经活动的时间演变，包括大脑在理解单个单词所需的数百毫秒内使用的时间相关的子过程，或者当单词一个接一个到达时，大脑用来构建和编码整个句子的意义的更复杂的过程，人们了解的更少。我们建议进行研究，并建立计算模型，研究在理解单个单词、短语、句子和故事时观察到的详细的空间和时间神经活动。这项拟议中的研究将特别针对以下问题：“在孤立理解一个单词所需的时间内，什么信息是由神经活动在何时何地编码的，以及大脑中的哪些子过程？”当一个新感知的单词首先激活感觉皮质，然后导致神经激活编码该单词的意思时，编码的信息流是什么？大脑如何将一个新遇到的词整合到句子或短语中较早的词的上下文中，以构成多词短语或句子的意义表征？以及“与孤立地处理相同的单词或作为非结构化集合(如{Kick，Joe，Ball})相比，语义期望和需求以及句法句法结构如何改变单词的处理？”为了研究这些问题，我们将(1)设计新的实验协议来探索在单词和句子处理过程中编码在神经信号中的信息流，(2)使用fMRI来收集新的大脑图像数据，以实现几毫米的空间分辨率，并使用脑磁图来实现几毫秒的时间分辨率，(3)开发和应用新的机器学习方法来建立计算模型，该计算模型集成并预测这些组合的实验数据。我们的目标是开发一个越来越准确的大脑如何理解单词、短语和句子的计算模型-一个对观察到的神经活动对新的语言刺激做出可测试预测的模型。智力价值：这项合作研究将先进的机器学习算法与脑磁图和功能磁共振成像的新实验协议结合在一起，以促进我们对人类大脑的两个基本悬而未决的问题的理解：大脑如何表示意义，以及什么神经认知过程从感知到的语言刺激逐个构建意义？更广泛的影响：如果成功，这项研究将影响广泛的社区，包括(1)认知神经科学和计算语言学，提供对大脑中语言处理的更好理解，(2)机器学习，通过推动开发新的时间序列和潜在变量分析方法，整合多个数据集，并纳入不同的 (3)大脑病理的临床研究，特别是与语言处理有关的临床研究，并为发育和后天语言障碍的治疗策略提供信息(4)通过在公共媒体上传播技术文章、教材和有关我们工作的新闻，对研究生、本科生和普通公众进行教育。

DESCRIPTION (provided by applicant): Over the past ten years a good deal has been learned from fMRI studies about the spatial patterns of neural activation used by the human brain to represent meanings of words and concepts. Much less is understood about the time evolution of this neural activity, including the temporally interrelated sub-processes the brain employs during the hundreds of milliseconds it takes to comprehend a single word, or the more complex processes it uses to construct and encode meaning of entire sentences as the words arrive one by one. We propose research to study, and to build computational models of, the detailed spatial and temporal neural activity observed during the comprehension of single words, phrases, sentences, and stories. This proposed research will specifically target the following questions: "What information is encoded by neural activity where and when, and by which subprocesses in the brain, during the time it takes to comprehend a single word in isolation?" "What is the flow of information encoded when a newly sensed word first activates sensory cortex, then later results in neural activation encoding the word meaning?" "How does the brain integrate a newly encountered word in the context of earlier words in the sentence or phrase, to compose the meaning representation of the multi-word phrase or sentence?" and "How do semantic expectations and demands, together with syntactic sentence structure alter the processing of words, compared to processing the same words in isolation, or as an unstructured set such as {kick, Joe, ball}?" To study these questions we will (1) devise novel experimental protocols to probe the flow of information encoded in neural signals during word and sentence processing, (2) collect new brain image data using both fMRI to achieve spatial resolution of a few millimeters, and MEG to achieve temporal resolution of a few milliseconds, (3) develop and apply novel machine learning approaches to build computational models that integrate and that predict this combined experimental data. Our goal is to develop an increasingly accurate computational model of how the brain comprehends words, phrases and sentences - a model that makes testable predictions about the neural activity observed in response to novel language stimuli. Intellectual Merit: This collaborative research brings together advanced machine learning algorithms with novel experimental protocols for MEG and fMRI brain imaging to advance our understanding of two fundamental open questions about the human brain: how does the brain represent meaning, and what neuro-cognitive processes construct that meaning piece-by-piece from perceived language stimuli? Broader Impacts: If successful, this research will impact a broad range of communities, including (1) cognitive neuroscience and computational linguistics, providing improved understanding of language processing in the brain, (2) machine learning, by driving the development of new methods for time series and latent variable analysis, integrating multiple data sets, and incorporating diverse background knowledge as priors, (3) clinical studies of brain pathologies, especially those related to language processing, and informing treatment strategies for developmental and acquired language disorders (4) education of graduates, undergraduates and the general public, through dissemination of technical articles, teaching materials, and news about our work in the public press.