Dynamic Neural Mechanisms of Audiovisual Speech Perception

项目摘要/摘要 言语感知本质上是多感官的：当我们与我们能看到的人交谈时， 我们的大脑结合了来自语音的听觉信息和来自面部的视觉信息。 言语感知是我们与他人互动的核心，因此也是我们的 最重要的认知能力。然而，我们对此的认识存在很大差距。 独一无二的人类技能，因为大多数可用于人类的实验技术都患有 时空分辨率差。为了弥补这一缺口，我们将检查神经 人类使用颅内记录(IEEG)进行视听语音感知的机制。 视听言语知觉发生在后上回和颞沟 (PSTG)在中尺度上了解pSTG内神经计算的动力学 (神经元被组织成柱状和斑块)在人类身上是不可能的。我们建议 利用快速变化的iEEG领域中的两项技术创新来研究它们 第一次：首先是高分辨率的颅内电极网，它允许从 皮质体积比标准iEEG栅格中的电极小数百倍；第二， 记录放置在非穿透性电极膜中的单个神经元活动的神经网格 在大脑皮层表面。我们的因果推理模型要求存在不同的听觉， 视觉和视听语音表示法。AIM 1将在 PSTG。目标2将研究pSTG中的低频振荡，以确定它们在 多感官言语感知。如果成功，AIMS将提供关于 多感官言语感知的神经机制，包括长期存在的谜团 视觉语言的知觉益处。

Project Summary/Abstract Speech perception is inherently multisensory: when conversing with someone that we can see, our brains combine auditory information from the voice with visual information from the face. Speech perception lies at the heart of our interactions with other people and is thus one of our most important cognitive abilities. However, there is a large gap in our knowledge about this uniquely human skill because most experimental techniques available in humans suffer from poor spatiotemporal resolution. In order to remedy this gap, we will examine the neural mechanisms of audiovisual speech perception using intracranial recording (iEEG) in humans. Audiovisual speech perception occurs in the posterior superior temporal gyrus and sulcus (pSTG) Understanding the dynamics of the neural computations within pSTG at the mesoscale (neurons organized into columns and patches) has been impossible in humans. We propose to leverage two technical innovations within the fast-changing field of iEEG to study them for the first time: first, high-resolution intracranial electrode grids, which allow for recording from a cortical volume hundreds of times smaller than the electrodes in standard iEEG grids; second, NeuroGrids that record single-neuron activity from a non-penetrating film of electrodes placed on the cortical surface. Our causal inference model requires the existence of distinct auditory, visual and audiovisual speech representations. Aim 1 will search for these representations in pSTG. Aim 2 will examine low-frequency oscillations in pSTG to determine their role in multisensory speech perception. If successful, the Aims will provide a comprehensive account of the neural mechanisms of multisensory speech perception, including the long-standing mystery of the perceptual benefit of visual speech.