AUDIOVISUAL SPEECH PROCESSING

描述(申请人提供)：拟议研究的目的是发展研究人类语音处理的新假说的能力，并获得支持该假说的初步结果。这一假设挑战了视听言语知觉研究中的主流观点，即在自下而上的感觉-知觉加工过程中，听觉和视觉语音信息很容易汇聚成一种共同的形式。需要检验的假设是，AV言语知觉对于言语知觉大脑来说是一个相对较晚的事件，遵循单峰加工，并涉及听觉和视觉言语表征之间的联系的皮质网络。为了研究这一假说，需要关于听觉和视觉语音刺激的皮层位置和加工潜伏期的结果。检验这一假设的方法将使用高分辨率的时间(事件相关电位-ERPs-通过电生理学获得)和空间(功能磁共振成像-fMRI)数据，这些数据来自听觉和视觉语音刺激的接受者。参与者将在视觉、听觉和视听条件下看到无意义的音节。匹配和不匹配的AV刺激将被测试，并且不匹配的刺激将根据它们的听觉到视觉的对应关系被定量地分级。在这项研究的驱动假说下，皮层反应被预测为对应关系的函数；因此，差异反应(它们的位置、时间动力学和强度)将被解释为听觉和视觉语音绑定(组合)过程的神经关联。该项目的一个主要重点将是根据偶极源和电流源密度对企业资源规划数据进行建模，每种类型的分析都考虑到功能和解剖核磁共振的结果。这个项目将导致一个研究计划，该计划将更详细地调查收敛或关联机制是否可以解释AV语音处理的问题。这是大脑如何处理多感官信息这一更大问题中的一个根本问题。所有涉及言语交流的临床应用都需要关于言语感知大脑的准确知识。

DESCRIPTION (provided by applicant): The purpose of the proposed research is to develop capabilities for studying a novel hypothesis concerning human speech processing and to obtain pilot results in support of the hypothesis. The hypothesis challenges the dominant view in audiovisual (AV) speech perception research, which is that auditory and visual speech information converges easily into a common format during bottom-up sensory-perceptual processing. The hypothesis to be examined is that AV speech perception is a relatively late event for the speech perceiving brain, following unimodal processing and involving cortical networks of associations between auditory and visual speech representations. In order to investigate this hypothesis, results are needed on cortical locations and processing latencies of auditory and visual speech stimuli. The approach to test the hypothesis will use high-resolution temporal (event-related potentials-ERPs - obtained with electrophysiology) and spatial (functional magnetic resonance imaging-fMRI) data obtained from perceivers presented with auditory and visual speech stimuli. Participants will be presented with nonsense syllables under visual-only, auditory-only, and AV conditions. Matched and mismatched AV stimuli wilt be tested and the mismatched stimuli will be quantitatively graded in terms of their auditory-to-visual correspondence. Under the driving hypothesis for the research, cortical responses are predicted to vary as a function of correspondence; therefore, differential responses (their locations, temporal dynamics, and strengths) will be interpreted as the neural correlates of the binding (combining) process for auditory and visual speech. A main focus of the project will be to model the ERP data in terms of dipole source and current source densities, each type of analysis taking into account results from functional and anatomical MRI. This project will lead to a research program that will investigate in more detail the issue of whether convergence or association mechanisms can account for AV speech processing. This is a fundamental issue within the larger question of how the brain processes multisensory information. Accurate knowledge about the speech perceiving brain is needed for all clinical applications involving speech communication.