Multisensory Processing of Human Speech Measured with msec and mm Resolution

以毫秒和毫米分辨率测量的人类语音的多感官处理

• 批准号: 9049247
• 负责人: DANIEL YOSHOR
• 金额: --
• 依托单位: MICHAEL E DEBAKEY VA MEDICAL CENTER
• 依托单位国家: 美国
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-04-01 至 2018-03-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/9049247
• 关键词: Acquired Deafness; Advisory Committees; Affect; Area; Attention; Auditory; Auditory area; Bayesian Modeling; Biology; Brain; Cerebral cortex; Communication; Coupling; Electrocorticogram; Epilepsy; Etiology; Face; Frequencies; Functional Magnetic Resonance Imaging; Functional disorder; Goals; Hearing; Human; Implanted Electrodes; Knowledge; Language; Lead; Measures; Methods; Modality; Modeling; Neurons; Noise; Patients; Perception; Phase; Process; Research; Resolution; Sensorineural Hearing Loss; Sensory; Services; Signal Transduction; Speech; Speech Perception; Stimulus; Stroke; Structure of superior temporal sulcus; Techniques; Testing; Therapeutic Intervention; Tinnitus; Translating; Traumatic Brain Injury; Veterans; Visual; Visual Cortex; Voice; abstracting; audiovisual speech; behavior measurement; central nervous system injury; cognitive ability; cognitive function; directed attention; disability; experience; extrastriate; extrastriate visual cortex; flexibility; hearing impairment; hemodynamics; human subject; improved; innovation; language impairment; millimeter; millisecond; multisensory; neuromechanism; novel therapeutics; public health relevance; relating to nervous system; response; targeted treatment; temporal measurement; therapy design; visual information; visual speech; visual stimulus

DESCRIPTION (provided by applicant): Project Summary/Abstract Speech perception is one of the most important cognitive functions of the brain. Humans use both the auditory information available in the heard speech and the visual information available from viewing the speaker's face to understand speech. Functional magnetic resonance imaging (fMRI) is the most popular method for examining human brain function, but suffers from the critical limitation of poor temporal resolution: because it measures hemodynamic changes, it has only ~sec resolution, rather than the ~ms timescale of neuronal activity. This renders fMRI a poor choice for examining dynamic neuronal processing. Instead, we propose to use electrocorticography (eCog), in which electrodes implanted in human subjects for the treatment of epilepsy are used to directly measure neuronal activity in the human brain. The major innovation of this proposal will be to use eCog to measure the dynamic neural processes underlying audiovisual speech perception. The central hypothesis is that rapid communication occurs between the human superior temporal sulcus (STS) and sensory areas through neural oscillations, specifically frequency-specific interactions between the multisensory cortex in the STS and earlier sensory areas in the auditory and visual cortex. As a measure of functional connectivity between areas in the speech network, we will measure trial-by-trial correlations in gamma-band power between brain areas. We assume that high correlations in gamma- band power reflect information transfer and a functional connection between the areas. Neuronal oscillations, particularly in the gamma range (~ 30 - 200 Hz), have been found to reflect neuronal spiking activity and several studies suggest that neuronal oscillations might be an important mechanism of information transfer in the brain that modulates spiking activity. We will test the hypothesis that gamma-band activity in the STS correlates with gamma-band activity in auditory association areas during multisensory speech perception. In ordinary speech, the auditory speech signal is much more informative than the visual speech signal. Therefore, we hypothesize that for clear audiovisual speech, the correlation in gamma oscillations between auditory cortex and STS is stronger than the correlation in gamma oscillations between visual cortex and STS. Conversely, we expect a stronger correlation between gamma-band activity in the visual cortex and the STS when the auditory speech signal is noisy. The proposed research will make use of direct neural recordings from the human brain to investigate whether the information content of the different speech signals affects the communication between early sensory areas and multisensory associative areas. These findings will help us understand the neuronal dynamics involved in speech perception and devise new therapies to help treat veterans with hearing loss and other disabilities.

描述（由申请人提供）： 言语感知是大脑最重要的认知功能之一。人类使用在听到的语音中可用的听觉信息和从观看说话者的面部可用的视觉信息来理解语音。功能性磁共振成像（fMRI）是检查人脑功能的最流行的方法，但受到时间分辨率差的严重限制：因为它测量的是时间分辨率。 血流动力学变化时，它只有~sec的分辨率，而不是神经元活动的~ms时间尺度。这使得功能磁共振成像检查动态神经元处理的一个穷人的选择。相反，我们建议使用皮质电描记术（eCog），其中植入人类受试者中用于治疗癫痫的电极用于直接测量人脑中的神经元活动。该提案的主要创新将是使用eCog来测量视听语音感知背后的动态神经过程。中心假设是，人类颞上沟（STS）和感觉区之间通过神经振荡发生快速通信，特别是STS中的多感觉皮层与听觉和视觉皮层中的早期感觉区之间的频率特异性相互作用。作为对言语网络中各区域之间功能连通性的测量，我们将测量大脑区域之间伽马波段功率的逐个试验相关性。我们假设伽马波段功率的高相关性反映了区域之间的信息传递和功能连接。已经发现神经元振荡，特别是在伽马范围（~30 - 200 Hz）中的神经元振荡反映神经元尖峰活动，并且若干研究表明神经元振荡可能是调节尖峰活动的大脑中的信息传递的重要机制。我们将测试的假设，在STS的γ-波段活动相关的γ-波段活动在听觉关联领域在多感官言语知觉。在普通语音中，听觉语音信号比视觉语音信号信息量大得多。因此，我们假设，对于清晰的视听语音，听觉皮层和STS之间的伽马振荡的相关性强于视觉皮层和STS之间的伽马振荡的相关性。相反，当听觉语音信号有噪声时，我们预计视觉皮层中的伽马波段活动与STS之间的相关性更强。这项研究将利用人类大脑的直接神经记录来研究不同语音信号的信息内容是否会影响早期感觉区域和多感觉关联区域之间的交流。这些发现将帮助我们了解参与言语感知的神经元动力学，并设计新的疗法来帮助治疗患有听力损失和其他残疾的退伍军人。