Dynamic Neural Mechanisms of Audiovisual Speech Perception

点击翻译按钮获取中文摘要

Project Summary/Abstract Speech perception is inherently multisensory: when conversing with someone that we can see, our brains combine auditory information from the voice with visual information from the face. Speech perception lies at the heart of our interactions with other people and is thus one of our most important cognitive abilities. However, there is a large gap in our knowledge about this uniquely human skill because most experimental techniques available in humans suffer from poor spatiotemporal resolution. In order to remedy this gap, we will examine the neural mechanisms of audiovisual speech perception using intracranial recording (iEEG) in humans. Audiovisual speech perception occurs in the posterior superior temporal gyrus and sulcus (pSTG) Understanding the dynamics of the neural computations within pSTG at the mesoscale (neurons organized into columns and patches) has been impossible in humans. We propose to leverage two technical innovations within the fast-changing field of iEEG to study them for the first time: first, high-resolution intracranial electrode grids, which allow for recording from a cortical volume hundreds of times smaller than the electrodes in standard iEEG grids; second, NeuroGrids that record single-neuron activity from a non-penetrating film of electrodes placed on the cortical surface. Our causal inference model requires the existence of distinct auditory, visual and audiovisual speech representations. Aim 1 will search for these representations in pSTG. Aim 2 will examine low-frequency oscillations in pSTG to determine their role in multisensory speech perception. If successful, the Aims will provide a comprehensive account of the neural mechanisms of multisensory speech perception, including the long-standing mystery of the perceptual benefit of visual speech.