AUDIOVISUAL SPEECH PROCESSING

描述（由申请人提供）：拟议研究的目的是开发研究有关人类语音处理的新假设的能力，并获得支持该假设的试点结果。这一假设挑战了视听语音感知研究中的主流观点，即听觉和视觉语音信息在自下而上的感觉知觉加工过程中很容易汇聚成一种共同的格式。要检验的假设是，对于语音感知大脑来说，AV语音感知是一个相对较晚的事件，在单峰处理之后，涉及听觉和视觉语音表征之间的关联皮层网络。为了验证这一假设，需要得到听觉和视觉言语刺激的皮层位置和处理潜伏期的结果。测试该假设的方法将使用高分辨率的时间（通过电生理学获得的事件相关电位- erp）和空间（功能性磁共振成像- fmri）数据，这些数据来自听觉和视觉语音刺激的感知者。参与者将在视觉、听觉和AV条件下看到无意义的音节。匹配和不匹配的AV刺激将被测试，不匹配的刺激将根据其听觉到视觉的对应程度进行定量分级。在该研究的驱动假说下，预测皮层反应作为对应的函数而变化；因此，不同的反应（它们的位置、时间动态和强度）将被解释为听觉和视觉语言结合过程的神经关联。该项目的主要重点将是根据偶极子源和电流源密度对ERP数据进行建模，每种类型的分析都考虑到功能性和解剖性MRI的结果。这个项目将导致一个研究项目，将更详细地调查是否收敛或关联机制可以解释AV语音处理的问题。这是大脑如何处理多感官信息这一更大问题中的一个基本问题。所有涉及言语交流的临床应用都需要对言语感知大脑的准确认识。

DESCRIPTION (provided by applicant): The purpose of the proposed research is to develop capabilities for studying a novel hypothesis concerning human speech processing and to obtain pilot results in support of the hypothesis. The hypothesis challenges the dominant view in audiovisual (AV) speech perception research, which is that auditory and visual speech information converges easily into a common format during bottom-up sensory-perceptual processing. The hypothesis to be examined is that AV speech perception is a relatively late event for the speech perceiving brain, following unimodal processing and involving cortical networks of associations between auditory and visual speech representations. In order to investigate this hypothesis, results are needed on cortical locations and processing latencies of auditory and visual speech stimuli. The approach to test the hypothesis will use high-resolution temporal (event-related potentials-ERPs - obtained with electrophysiology) and spatial (functional magnetic resonance imaging-fMRI) data obtained from perceivers presented with auditory and visual speech stimuli. Participants will be presented with nonsense syllables under visual-only, auditory-only, and AV conditions. Matched and mismatched AV stimuli wilt be tested and the mismatched stimuli will be quantitatively graded in terms of their auditory-to-visual correspondence. Under the driving hypothesis for the research, cortical responses are predicted to vary as a function of correspondence; therefore, differential responses (their locations, temporal dynamics, and strengths) will be interpreted as the neural correlates of the binding (combining) process for auditory and visual speech. A main focus of the project will be to model the ERP data in terms of dipole source and current source densities, each type of analysis taking into account results from functional and anatomical MRI. This project will lead to a research program that will investigate in more detail the issue of whether convergence or association mechanisms can account for AV speech processing. This is a fundamental issue within the larger question of how the brain processes multisensory information. Accurate knowledge about the speech perceiving brain is needed for all clinical applications involving speech communication.