Do you see what you are hearing? Cross-modal effects of speech sounds on lipreading
Do you see what you are hearing? Cross-modal effects of speech sounds on lipreading
复制标题
DOI：
10.1016/j.neulet.2010.01.019
复制
发表时间：
2010-03-03
影响因子：
2.5
通讯作者：
Vroomen, Jean
中科院分区：
文献类型：
--
作者：
Baart, Martijn;Vroomen, Jean
众所周知，从嘴部运动导出的视觉信息（即，唇读）可以对听觉语音识别产生深远的影响（例如McGurk效应[16]）。在这里，我们研究了相反的现象，即听觉言语是否影响唇读。我们报告说，语音声音配音到lipread语音影响lipread令牌的即时识别。这种效果可能反映了真正的跨模态整合的感觉信号，而不仅仅是一个简单的反应偏差，因为我们还观察到自适应变化的模糊lipread令牌暴露后，不一致的视听适配器刺激的视觉识别。据推测，听众已经学会了根据声音来标记唇读刺激，从而证明听觉和唇读之间的相互作用是真正的双向的。(C)2010爱思唯尔爱尔兰有限公司版权所有。
It is well known that visual information derived from mouth movements (i.e., lipreading) can have profound effects on auditory speech identification (e.g. the McGurk-effect [16]). Here we examined the reverse phenomenon, namely whether auditory speech affects lipreading. We report that speech sounds dubbed onto lipread speech affect immediate identification of lipread tokens. This effect likely reflects genuine cross-modal integration of sensory signals and not just a simple response bias because we also observed adaptive shifts in visual identification of the ambiguous lipread tokens after exposure to incongruent audiovisual adapter stimuli. Presumably, listeners had learned to label the lipread stimulus in accordance with the sound, thus demonstrating that the interaction between hearing and lipreading is genuinely bi-directional. (C) 2010 Elsevier Ireland Ltd. All rights reserved.