Dynamic Neural Mechanisms of Audiovisual Speech Perception

摘要-（题目：视听言语知觉的动态神经机制） 自然的言语感知是多感官的;当与我们能看到的人交谈时，我们的大脑 将来自面部、姿势和手势的视觉（V）信息与来自面部、姿势和手势的听觉（A）信息联合收割机。 声音底层的语音处理非常迅速，输入的AV单元（例如，音节）到达 每隔几百毫秒，必须在下一个音节到达之前进行编码和传递。最后， 这种自下而上的感觉信息与自上而下的认知成分相结合：我们所感知的是 深受其背景的影响。语言是人类的基本特征，因此，它的大脑机制通常是 用非侵入性功能磁共振成像、脑电图和脑磁图进行研究。由于每种方法在空间或 时间分辨率，识别言语感知的特定大脑机制- AV整合，精确 以及快速信息编码和自上而下的控制-几乎是一个棘手的问题。 这个为期三年的U 01项目将通过直接记录神经元集合来回避这个问题 （皮层电图或ECoG）活动和单个神经元活动，沿着直接刺激选择的 手术癫痫患者大脑中处理AV语音的部位。我们的协作ECoG团队 体现了多感官整合和言语感知方面的专业知识，并利用了这些技能， 神经科学家、神经外科医生、工程师、神经心理学家、神经学家和伦理学家的观点 在三个领先的癫痫中心：哥伦比亚大学医学中心，贝勒医学院和 长岛北岸犹太医疗中心通过结合三个医院的专业知识和病人 中心，我们将能够解决个人调查人员无法解决的问题。 基于我们过去的工作，并得到初步数据的支持，我们的总体假设是， 神经元兴奋性的波动-振荡-在语音处理中起关键作用。 目的1测试一般假设，即δ/θ范围（2-8 Hz）的神经元振荡在神经元振荡中起关键作用。 听觉和视觉语音信息的整合。 目的2检验高频活动（50 Hz及以上）编码以下表征的一般假设： 听觉和视觉语音信息，反映了自下而上和自上而下的感知影响。 振荡动力学作为机械工具（目的1）的这一建议中采用的概念， 在动态自上而下的控制下组织神经元放电模式中的信息编码（目标2） 这是言语科学范式转变的一部分该提案的总体目标是为建立一个 为这个新的范例，并为全面了解大脑回路奠定基础， 生理过程的自然语音感知，包括复杂的社会环境。

ABSTRACT – (Title: Dynamic Neural Mechanisms of Audiovisual Speech Perception) Natural speech perception is multisensory; when conversing with someone that we can see, our brains combine visual (V) information from face, postural and hand gestures with auditory (A) information from the voice. The underlying speech processing is extremely rapid, with incoming AV units (e.g., syllables) arriving every few hundred milliseconds that must be encoded and passed on before the next syllable arrives. Finally, this bottom-up sensory information is combined with a top-down cognitive component: what we perceive is strongly influenced by its context. Speech is fundamentally human, and thus, its brain mechanisms are usually studied with noninvasive fMRI, EEG and MEG. Because each method has critical limitations in spatial or temporal resolution, identifying the specific brain mechanisms of speech perception - AV integration, precise and rapid information encoding and top-down control - is a nearly intractable problem. This three-year U01 project will sidestep the problem using direct recording of neuron ensemble (electrocorticographic or ECoG) activity and single neuron activity, along with direct stimulation of selected sites in the brains of surgical epilepsy patients as they process AV speech. Our collaborative ECoG team embodies expertise in multisensory integration and speech perception and leverages the skills and perspectives of neuroscientists, neurosurgeons, engineers, neuropsychologists, neurologists, and ethicists across three leading epilepsy centers: Columbia University Medical Center, Baylor College of Medicine and Northshore-Long Island Jewish Medical Center. By combining the expertise and patients available at all three centers, we will be able to tackle problems that are inaccessible to individual investigators. Our overarching hypothesis, building on our past work and supported by preliminary data, is that fluctuations in the excitability of neurons—oscillations—play a key role in speech processing. Aim 1 tests the general hypothesis that delta/theta range (2-8 Hz) neuronal oscillations play a key role in the integration of auditory and visual speech information. Aim 2 tests the general hypothesis that high-frequency activity (50 Hz and above) encodes representations of auditory and visual speech information, reflecting both bottom-up and top-down influences on perception. The concept employed in this proposal of oscillatory dynamics as mechanistic instruments (Aim 1) that organize the encoding of information in neuronal firing patterns under dynamic top-down control (Aim 2) are part of a paradigm shift in speech science. The broad goal of this proposal is to contribute key foundations for this new paradigm, and set the stage for a comprehensive understanding of the brain circuits and physiological processes underlying natural speech perception, including complex social settings.