Dynamic Neural Mechanisms of Audiovisual Speech Perception

项目概要/摘要 言语感知本质上是多感官的：当与我们能看到的人交谈时， 我们的大脑将来自声音的听觉信息与来自面部的视觉信息联合收割机结合起来。 言语感知是我们与他人互动的核心，因此是我们的一个重要因素。 最重要的认知能力。然而，我们对此的认识还有很大的差距 这是人类独有的技能，因为大多数可用于人类的实验技术都存在以下问题： 时空分辨率差。为了弥补这一差距，我们将检查神经 在人类中使用颅内记录（iEEG）的视听言语感知机制。 视听言语知觉发生在后上级颞回和沟 （pSTG）在中尺度上理解pSTG中的神经计算动态 （神经元组织成柱状和片状）在人类中是不可能的。我们建议 利用快速变化的iEEG领域中的两项技术创新， 首次：首先，高分辨率颅内电极网格，允许从 皮质体积比标准iEEG网格中的电极小数百倍;第二， NeuroGrids记录单神经元的活动从一个非穿透膜的电极放置 在皮层表面。我们的因果推理模型需要存在不同的听觉， 视觉和视听语音表示。Aim 1将在 pSTG。目标2将检查pSTG中的低频振荡，以确定它们在以下方面的作用： 多感官言语知觉如果成功，目标将提供一个全面的帐户， 多感官言语感知的神经机制，包括长期存在的谜团 视觉语言的感知优势。

Project Summary/Abstract Speech perception is inherently multisensory: when conversing with someone that we can see, our brains combine auditory information from the voice with visual information from the face. Speech perception lies at the heart of our interactions with other people and is thus one of our most important cognitive abilities. However, there is a large gap in our knowledge about this uniquely human skill because most experimental techniques available in humans suffer from poor spatiotemporal resolution. In order to remedy this gap, we will examine the neural mechanisms of audiovisual speech perception using intracranial recording (iEEG) in humans. Audiovisual speech perception occurs in the posterior superior temporal gyrus and sulcus (pSTG) Understanding the dynamics of the neural computations within pSTG at the mesoscale (neurons organized into columns and patches) has been impossible in humans. We propose to leverage two technical innovations within the fast-changing field of iEEG to study them for the first time: first, high-resolution intracranial electrode grids, which allow for recording from a cortical volume hundreds of times smaller than the electrodes in standard iEEG grids; second, NeuroGrids that record single-neuron activity from a non-penetrating film of electrodes placed on the cortical surface. Our causal inference model requires the existence of distinct auditory, visual and audiovisual speech representations. Aim 1 will search for these representations in pSTG. Aim 2 will examine low-frequency oscillations in pSTG to determine their role in multisensory speech perception. If successful, the Aims will provide a comprehensive account of the neural mechanisms of multisensory speech perception, including the long-standing mystery of the perceptual benefit of visual speech.