Audio-Visual interaction in speech perception; a PET study

人脑在感知外部信号的过程中，有效地整合了来自多个感官通道的信息。这种多模式处理有利于比单模式处理更快和更准确地识别信息。在言语感知中，说话人面部的视觉信息可以有效地用于言语感知。在发声时看到一致的视觉信息(面部运动)可以提高听觉语音对背景噪声的感知能力。即使呈现的视觉提示与听觉输入不一致，上述视听意图也可能发生。在典型情况下，看到不一致的发音手势可以从语音上改变听觉感知(McGurk效应)。在本研究中，我们使用正电子发射计算机断层扫描(PET)来识别在呈现不一致的视听语音信号时大脑的激活。当声学/BE/一起呈现在…上时，所有受试者回答/BE/的行为测量更多关于视觉/BE/的信息。另一方面，当声学/BE/呈现视觉/GE/(不一致条件)时，12个受试者中有9个被回答/De/(听觉错觉)。脑激活对于观察到听觉错觉的9名受试者，使用基于体素的图像减影进行PET分析。当将/BE/-/Ge/的A-V刺激条件与/BE/(A)-/BE/(V)的A-V刺激或/BE/-/VN/的A-V刺激条件进行比较时，在Amvadala、扣带回前部和尾状头部观察到激活区(p&gt；0.005未校正)。Sekiyama等人(1994年)指出，许多受试者(特别是日本受试者)在McGurk刺激中看到不一致的A-V组合时，会经历一种不愉快或奇怪的感觉。在不一致的A-V组合(/be/-/ge/)的表现中观察到的边缘系统区域的激活可能与这种对几乎没有经验的不寻常刺激的情绪反应有关。较少

Human brain integrates information from multisensory modalities effectively during perceiving external signals. This mutimodal processing is beneficial for faster and more accurate recognition of information than unimodal processing. In speech perception, it is well known that visual information from speaker's face can be effectively utilixed in speech perception. Seeing consistent visual information (facial movement) in phonation can improve the perceptibility in hearing speech sound against background noise. Even when presented visual cue is inconsistent with auditory input, above-mentioned audio-visual intention can occur. In typical case, seeing incongruent phonatory gesture can modify auditory percept phonetically (McGurk effects). In the present study, we used PET to identify brain activations during the presentation of inconsistent audiovisual phonetic signals.Results and Discussion1. Behavioral measures All the subjects answered /be/ when the acoustic /be/ was presented togethe … More r with visual /be/. On the other hand, nine of 12 subjects were answered /de/ (auditory illusion), when the acoustic /be/ was presented with visual /ge/ (inconsistent condition).2. Brain activationFor the nine subjects in whom auditory illusion were observed, PET analysis was conducted using the voxel based image subtraction. When the condition of A-V stimulation of /be/-/ge/ was compared with that of A-V stimulation of /be/(A)-/be/(V), or A-V stimulation of /be/-/VN/, activated areas were observed in the amvadala, the anteriorpart of the cingulate gyrus, and the caudate head (p>0.005 uncorrected).Sekiyama et al (1994) indicated that many of subjects (especially in Japanese subjects) experienced a kind of unpleasant or strange feeling when they see the inconsistent A-V combination in McGurk stimulus. The activation of regions in the limbic system observed in the presentation of inconsistent A-V combination (/be/-/ge/) might be related to this emotional reaction for the unusual stimulus that is hardly experienced. Less