Visual information constrains early and late stages of spoken-word recognition in sentence context
Visual information constrains early and late stages of spoken-word recognition in sentence context
复制标题
DOI：
10.1016/j.ijpsycho.2013.06.016
复制
发表时间：
2013-07-01
影响因子：
3
通讯作者：
Soto-Faraco, Salvador
中科院分区：
文献类型：
--
作者：
Brunelliere, Angele;Sanchez-Garcia, Carolina;Soto-Faraco, Salvador
视听言语知觉的研究主要考虑音素、音节和词的加工水平。在这里，我们研究了视觉语音信息在识别自然句子上下文中嵌入的单词时可能施加的限制。我们记录了一些词的事件相关电位(ERPs)，这些词可以根据先前的语义句子上下文进行强预测或弱预测，并且其初始音素因嘴唇运动而产生的视觉显著程度不同。当句子被视听呈现时(实验1)，与强可预测词相比，从语义语境中弱预测的词产生了更大的长期持续的N400。这种语义效果与N400后期的视觉显著程度相互作用。当比较视听和听觉单独呈现时(实验2)，在视听通道中观察到典型的听觉诱发N100反应的减幅效应。有趣的是，在早期的N100反应和后期的N400时间窗中，出现了高视觉显著限制和低视觉显著限制的特定益处，这证实了实验1的结果。综上所述，我们的结果表明，视觉语音的显著可以在相对较晚的阶段对听觉加工和单词识别产生影响，从而表明在自然语音理解过程中，视听整合与信息加工的其他(可以说是更高的)阶段之间存在着强烈的交互作用。(C)2013爱思唯尔B.V.保留所有权利。
Audiovisual speech perception has been frequently studied considering phoneme, syllable and word processing levels. Here, we examined the constraints that visual speech information might exert during the recognition of words embedded in a natural sentence context. We recorded event-related potentials (ERPs) to words that could be either strongly or weakly predictable on the basis of the prior semantic sentential context and, whose initial phoneme varied in the degree of visual saliency from lip movements. When the sentences were presented audio-visually (Experiment 1), words weakly predicted from semantic context elicited a larger long-lasting N400, compared to strongly predictable words. This semantic effect interacted with the degree of visual saliency over a late part of the N400. When comparing audio-visual versus auditory alone presentation (Experiment 2), the typical amplitude-reduction effect over the auditory-evoked N100 response was observed in the audiovisual modality. Interestingly, a specific benefit of high- versus low-visual saliency constraints occurred over the early N100 response and at the late N400 time window, confirming the result of Experiment 1. Taken together, our results indicate that the saliency of visual speech can exert an influence over both auditory processing and word recognition at relatively late stages, and thus suggest strong interactivity between audio-visual integration and other (arguably higher) stages of information processing during natural speech comprehension. (C) 2013 Elsevier B.V. All rights reserved.