CRCNS: Cortical representation of phonetic, syntactic and semantic information during speech perception and language comprehension

该项目的首要目标是发现与语言相关的信息如何在人脑中表示和处理。为了解决这个问题，我们建议使用一种新的计算建模方法，体素建模。体素建模借鉴了非线性系统识别的原理，它提供了一种有效的方法，可以使用在自然条件下收集的复杂数据集来测试关于语言表示的多个假设。具体的研究计划分为三个目标，每个目标针对不同形式的语言相关信息。目标1将揭示语音的低层次特征，如频谱功率，频谱调制和音素结构，是如何在人类皮层中表现的。受试者将被动地听人类讲话，同时通过功能性MRI记录血液动力学脑活动。然后将使用逐体素建模来确定大脑中的每个点（即，每个体素或体积像素）被调谐用于这些不同的特征。使用类似的方法，目标2将揭示句法和语义特征是如何在大脑皮层中表达的。最后，目标3将揭示如何语言相关的信息表示时，它是由听觉和视觉形式。在这种情况下，将使用语音和视频刺激。将针对听觉和视觉刺激期间记录的数据估计单独的模型，并将跨模态比较体素调谐。根据这一建议开发的体素计算模型将揭示这些不同类型的语言相关信息是如何在皮层表面表示的。这些模型还将提供关于大脑如何对新的语音刺激做出反应的明确预测。这项研究的结果将对与言语感知和产生相关的临床问题产生广泛的影响，它们可能成为强大的大脑解码设备的基础，使神经系统患者能够仅通过思想进行交流。

The overarching goal of this project is to discover how language-related information is represented and processed in the human brain. To address this issue we propose to use a novel computational modeling approach, voxel-wise modeling. Voxel-wise modeling draws from the principles of nonlinear system identification, and it provides an efficient method for using complex data sets collected under naturalistic conditions to test multiple hypotheses about language representation. The specific research plan is divided into three aims, each targeted at a different form of language-related information. Aim 1 will reveal how low-level features of speech, such as spectral power, spectral modulation and phonemic structure, are represented across human cortex. Subjects will passively listen to human speech while hemodynamic brain activity is recorded by functional MRI. Voxel-wise modeling will then be used to determine how each point in the brain (i.e., each voxel, or volumetric pixel) is tuned for these various features. Using analogous methods, Aim 2 will reveal how syntactic and semantic features are represented across cortex. Finally, Aim 3 will reveal how language-related information is represented when it is delivered by auditory versus visual modalities. In this case speech and video stimuli will be used. Separate models will be estimated for data recorded during auditory and visual stimulation, and voxel-wise tuning will be compared across modalities. The voxel-wise computational models developed under this proposal will reveal how these various types of language-related information are represented across the cortical surface. These models will also provide clear predictions about how the brain will respond to novel speech stimuli. The results of the proposed research will have broad impacts on clinical problems related to speech perception and production, and they could form the basis of powerful brain decoding device that would enable neurological patients to communicate by thought alone.