Audio-Visual interaction in speech perception; a PET study

人类大脑在感知外部信号的过程中，能够有效地整合多感官信息。与单模态处理相比，这种多模态处理有利于更快、更准确地识别信息。在语音感知中，说话人的面部视觉信息可以有效地用于语音感知。在发音中看到一致的视觉信息（面部运动）可以提高对背景噪音的听觉感知能力。即使呈现的视觉线索与听觉输入不一致，也会出现上述视听意图。在典型情况下，看到不一致的发音手势可以在语音上改变听觉感知（McGurk效应）。在本研究中，我们使用PET来识别在呈现不一致的视听语音信号时的大脑激活。结果与讨论行为测试所有的被试都回答了/be/当声音/be/和视觉/be/一起出现时。另一方面，当声学/be/呈现视觉/ge/（不一致条件）时，12名受试者中有9人回答/de/（幻听）。对观察到听错觉的9名受试者，采用基于体素的图像减法进行PET分析。将/be/-/ge/的A-V刺激与/be/(A)-/be/(V)的A-V刺激、/be/-/VN/的A-V刺激进行比较，在amvadala、扣带回前部和尾状头中观察到激活区（p>0.005未校正）。Sekiyama等（1994）指出，许多被试（尤其是日本被试）在看到McGurk刺激中不一致的a - v组合时，会产生一种不愉快或奇怪的感觉。在不一致的A-V组合（/be/-/ge/）中观察到的边缘系统区域的激活可能与这种对罕见刺激的情绪反应有关。少

Human brain integrates information from multisensory modalities effectively during perceiving external signals. This mutimodal processing is beneficial for faster and more accurate recognition of information than unimodal processing. In speech perception, it is well known that visual information from speaker's face can be effectively utilixed in speech perception. Seeing consistent visual information (facial movement) in phonation can improve the perceptibility in hearing speech sound against background noise. Even when presented visual cue is inconsistent with auditory input, above-mentioned audio-visual intention can occur. In typical case, seeing incongruent phonatory gesture can modify auditory percept phonetically (McGurk effects). In the present study, we used PET to identify brain activations during the presentation of inconsistent audiovisual phonetic signals.Results and Discussion1. Behavioral measures All the subjects answered /be/ when the acoustic /be/ was presented togethe … More r with visual /be/. On the other hand, nine of 12 subjects were answered /de/ (auditory illusion), when the acoustic /be/ was presented with visual /ge/ (inconsistent condition).2. Brain activationFor the nine subjects in whom auditory illusion were observed, PET analysis was conducted using the voxel based image subtraction. When the condition of A-V stimulation of /be/-/ge/ was compared with that of A-V stimulation of /be/(A)-/be/(V), or A-V stimulation of /be/-/VN/, activated areas were observed in the amvadala, the anteriorpart of the cingulate gyrus, and the caudate head (p>0.005 uncorrected).Sekiyama et al (1994) indicated that many of subjects (especially in Japanese subjects) experienced a kind of unpleasant or strange feeling when they see the inconsistent A-V combination in McGurk stimulus. The activation of regions in the limbic system observed in the presentation of inconsistent A-V combination (/be/-/ge/) might be related to this emotional reaction for the unusual stimulus that is hardly experienced. Less