The processing of audio-visual speech: empirical and neural bases
The processing of audio-visual speech: empirical and neural bases
复制标题
DOI：
10.1098/rstb.2007.2155
复制
发表时间：
2008-03-12
影响因子：
6.3
通讯作者：
Campbell, Ruth
中科院分区：
文献类型：
--
作者：
Campbell, Ruth
在这篇选择性的回顾中，我概述了看到说话者影响言语听觉感知的几种方式，包括但不限于麦格克效应。到目前为止，研究表明，所有的语言水平都容易受到视觉的影响，并且可以描述两种主要的加工模式：一种是互补模式，在这种模式下，视觉比听觉更有效地为语音流中某些未指定的部分提供信息；另一种是相关模式，在这种模式下，视觉部分复制了关于动态发音模式的信息。视听言语的皮层关联表明，在神经和知觉水平上，言语的听觉加工都受到视觉的影响，因此听觉言语区域被看见的言语激活。自然语言的处理，无论它是被听到、看到还是听到和看到，都激活了周边语区域(左和右)。激活很可能以特定的顺序发生。首先，激活颞上区，然后是顶下区，最后是额下区(左、右)。视觉输入流与核心的周边语系统存在一定的差异，提示互补的声表面信息对视觉腹侧加工流有特殊的作用，而对于相关视觉语音，对视觉运动敏感的背侧加工流可能相对更多地参与其中。
In this selective review, I outline a number of ways in which seeing the talker affects auditory perception of speech, including, but not confined to, the McGurk effect. To date, studies suggest that all linguistic levels are susceptible to visual influence, and that two main modes of processing can be described: a complementary mode, whereby vision provides information more efficiently than hearing for some under-specified parts of the speech stream, and a correlated mode, whereby vision partially duplicates information about dynamic articulatory patterning.Cortical correlates of seen speech suggest that at the neurological as well as the perceptual level, auditory processing of speech is affected by vision, so that 'auditory speech regions' are activated by seen speech. The processing of natural speech, whether it is heard, seen or heard and seen, activates the perisylvian language regions (left > right). It is highly probable that activation occurs in a specific order. First, superior temporal, then inferior parietal and finally inferior frontal regions (left > right) are activated. There is some differentiation of the visual input stream to the core perisylvian language system, suggesting that complementary seen speech information makes special use of the visual ventral processing stream, while for correlated visual speech, the dorsal processing stream, which is sensitive to visual movement, may be relatively more involved.