Multisensory Processing of Human Speech Measured with msec and mm Resolution

描述(由申请人提供)： 项目摘要/摘要言语感知是大脑最重要的认知功能之一。人类通过观察说话人的脸部，既可以利用听觉信息，也可以利用视觉信息来理解言语。功能磁共振成像(Fmri)是检查人类大脑功能最流行的方法，但它受到时间分辨率低的严重限制：因为它测量 血流动力学改变，它只有~秒的分辨率，而不是神经元活动的~ms时间尺度。这使得fMRI成为检查动态神经元处理的一个糟糕的选择。相反，我们建议使用皮层脑电图术(ECOG)，即在人类受试者体内植入用于治疗癫痫的电极，用于直接测量人类大脑中的神经元活动。这一建议的主要创新将是使用ECOG来测量视听言语知觉背后的动态神经过程。中心假说是，人类的颞上沟(STS)和感觉区之间通过神经振荡发生快速通信，特别是在STS的多感觉皮层和听觉和视觉皮质的早期感觉区域之间的特定频率的相互作用。作为语音网络区域之间功能连通性的衡量标准，我们将测量大脑区域之间伽马频段功率的逐次试验相关性。我们假设伽马频段功率的高度相关性反映了信息的传递和区域之间的功能联系。神经元振荡，特别是在伽马范围(~30-200 Hz)，已经被发现反映神经元的放电活动，一些研究表明神经元振荡可能是大脑中调节放电活动的信息传递的重要机制。我们将检验这一假设，即STS中的伽马频段活动与多感官言语知觉过程中听觉关联区的伽马频段活动相关。在普通语音中，听觉语音信号比视觉语音信号信息量大得多。因此，我们假设对于清晰的视听语音，听觉皮层和STS之间的伽马振荡的相关性要强于视觉皮质和STS之间的伽马振荡的相关性。相反，当听觉语音信号有噪声时，我们预计视觉皮质中的伽马频段活动与STS之间有更强的相关性。这项拟议的研究将利用来自人脑的直接神经记录来调查不同语音信号的信息内容是否会影响早期感觉区域和多感觉关联区域之间的交流。这些发现将帮助我们了解涉及言语感知的神经动力学，并设计新的疗法来帮助治疗患有听力损失和其他残疾的退伍军人。

DESCRIPTION (provided by applicant): Project Summary/Abstract Speech perception is one of the most important cognitive functions of the brain. Humans use both the auditory information available in the heard speech and the visual information available from viewing the speaker's face to understand speech. Functional magnetic resonance imaging (fMRI) is the most popular method for examining human brain function, but suffers from the critical limitation of poor temporal resolution: because it measures hemodynamic changes, it has only ~sec resolution, rather than the ~ms timescale of neuronal activity. This renders fMRI a poor choice for examining dynamic neuronal processing. Instead, we propose to use electrocorticography (eCog), in which electrodes implanted in human subjects for the treatment of epilepsy are used to directly measure neuronal activity in the human brain. The major innovation of this proposal will be to use eCog to measure the dynamic neural processes underlying audiovisual speech perception. The central hypothesis is that rapid communication occurs between the human superior temporal sulcus (STS) and sensory areas through neural oscillations, specifically frequency-specific interactions between the multisensory cortex in the STS and earlier sensory areas in the auditory and visual cortex. As a measure of functional connectivity between areas in the speech network, we will measure trial-by-trial correlations in gamma-band power between brain areas. We assume that high correlations in gamma- band power reflect information transfer and a functional connection between the areas. Neuronal oscillations, particularly in the gamma range (~ 30 - 200 Hz), have been found to reflect neuronal spiking activity and several studies suggest that neuronal oscillations might be an important mechanism of information transfer in the brain that modulates spiking activity. We will test the hypothesis that gamma-band activity in the STS correlates with gamma-band activity in auditory association areas during multisensory speech perception. In ordinary speech, the auditory speech signal is much more informative than the visual speech signal. Therefore, we hypothesize that for clear audiovisual speech, the correlation in gamma oscillations between auditory cortex and STS is stronger than the correlation in gamma oscillations between visual cortex and STS. Conversely, we expect a stronger correlation between gamma-band activity in the visual cortex and the STS when the auditory speech signal is noisy. The proposed research will make use of direct neural recordings from the human brain to investigate whether the information content of the different speech signals affects the communication between early sensory areas and multisensory associative areas. These findings will help us understand the neuronal dynamics involved in speech perception and devise new therapies to help treat veterans with hearing loss and other disabilities.