CRCNS: Information Flow in the Brain During Language and Meaning Comprehension

描述（由申请人提供）：在过去的十年中，从功能磁共振成像研究中了解到了很多关于人脑用于表示单词和概念含义的神经激活的空间模式。关于这种神经活动的时间演变，包括大脑在理解一个单词所需的数百毫秒内所使用的时间相关的子过程，或者当单词一个接一个到达时，它用于构建和编码整个句子的含义的更复杂的过程，我们了解得更少。我们建议研究，并建立计算模型，在理解单个单词，短语，句子和故事的过程中观察到的详细的空间和时间神经活动。这项拟议中的研究将专门针对以下问题：“在孤立地理解一个单词所需的时间内，什么信息在何时何地由神经活动编码，以及由大脑中的哪些子过程编码？“当一个新感觉到的单词首先激活感觉皮层，然后导致神经激活编码单词含义时，编码的信息流是什么？大脑是如何将一个新遇到的单词整合到句子或短语中先前单词的上下文中，以组成多词短语或句子的意义表征的？与孤立地或作为非结构化集合（如{kick，Joe，ball}）处理相同的单词相比，语义期望和需求以及句法句子结构如何改变单词的处理？“为了研究这些问题，我们将（1）设计新的实验方案，以探测在单词和句子处理过程中神经信号中编码的信息流，（2）收集新的大脑图像数据，使用fMRI实现几毫米的空间分辨率，并使用MEG实现几毫秒的时间分辨率，（3）开发和应用新的机器学习方法来构建整合和预测这些组合实验数据的计算模型。我们的目标是开发一个越来越精确的计算模型，来研究大脑如何翻译单词、短语和句子--这个模型可以对新的语言刺激所观察到的神经活动做出可测试的预测。智力优势：这项合作研究将先进的机器学习算法与MEG和fMRI脑成像的新实验方案结合在一起，以促进我们对人类大脑两个基本开放问题的理解：大脑如何表示意义，以及什么样的神经认知过程从感知的语言刺激中逐段构建意义？更广泛的影响：如果成功，这项研究将影响广泛的社区，包括（1）认知神经科学和计算语言学，提供对大脑语言处理的更好理解，（2）机器学习，通过推动时间序列和潜在变量分析新方法的开发，整合多个数据集，并将不同的 背景知识作为先验，（3）大脑病理学的临床研究，特别是与语言处理相关的研究，并为发育性和后天语言障碍的治疗策略提供信息（4）通过传播技术文章，教学材料和公共媒体上有关我们工作的新闻，对研究生，本科生和公众进行教育。

DESCRIPTION (provided by applicant): Over the past ten years a good deal has been learned from fMRI studies about the spatial patterns of neural activation used by the human brain to represent meanings of words and concepts. Much less is understood about the time evolution of this neural activity, including the temporally interrelated sub-processes the brain employs during the hundreds of milliseconds it takes to comprehend a single word, or the more complex processes it uses to construct and encode meaning of entire sentences as the words arrive one by one. We propose research to study, and to build computational models of, the detailed spatial and temporal neural activity observed during the comprehension of single words, phrases, sentences, and stories. This proposed research will specifically target the following questions: "What information is encoded by neural activity where and when, and by which subprocesses in the brain, during the time it takes to comprehend a single word in isolation?" "What is the flow of information encoded when a newly sensed word first activates sensory cortex, then later results in neural activation encoding the word meaning?" "How does the brain integrate a newly encountered word in the context of earlier words in the sentence or phrase, to compose the meaning representation of the multi-word phrase or sentence?" and "How do semantic expectations and demands, together with syntactic sentence structure alter the processing of words, compared to processing the same words in isolation, or as an unstructured set such as {kick, Joe, ball}?" To study these questions we will (1) devise novel experimental protocols to probe the flow of information encoded in neural signals during word and sentence processing, (2) collect new brain image data using both fMRI to achieve spatial resolution of a few millimeters, and MEG to achieve temporal resolution of a few milliseconds, (3) develop and apply novel machine learning approaches to build computational models that integrate and that predict this combined experimental data. Our goal is to develop an increasingly accurate computational model of how the brain comprehends words, phrases and sentences - a model that makes testable predictions about the neural activity observed in response to novel language stimuli. Intellectual Merit: This collaborative research brings together advanced machine learning algorithms with novel experimental protocols for MEG and fMRI brain imaging to advance our understanding of two fundamental open questions about the human brain: how does the brain represent meaning, and what neuro-cognitive processes construct that meaning piece-by-piece from perceived language stimuli? Broader Impacts: If successful, this research will impact a broad range of communities, including (1) cognitive neuroscience and computational linguistics, providing improved understanding of language processing in the brain, (2) machine learning, by driving the development of new methods for time series and latent variable analysis, integrating multiple data sets, and incorporating diverse background knowledge as priors, (3) clinical studies of brain pathologies, especially those related to language processing, and informing treatment strategies for developmental and acquired language disorders (4) education of graduates, undergraduates and the general public, through dissemination of technical articles, teaching materials, and news about our work in the public press.