Audio-Visual interaction in speech perception; a PET study

语音感知中的视听交互；

• 批准号: 13671764
• 负责人: KAWASE Tetsuaki
• 金额: $ 2.3万
• 依托单位: Tohoku University
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (C)
• 财政年份: 2001
• 资助国家: 日本
• 起止时间: 2001 至 2002
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-13671764/
• 关键词: Mutimodal processing; speech recognition; integration; McGurk effect; PET

Human brain integrates information from multisensory modalities effectively during perceiving external signals. This mutimodal processing is beneficial for faster and more accurate recognition of information than unimodal processing. In speech perception, it is well known that visual information from speaker's face can be effectively utilixed in speech perception. Seeing consistent visual information (facial movement) in phonation can improve the perceptibility in hearing speech sound against background noise. Even when presented visual cue is inconsistent with auditory input, above-mentioned audio-visual intention can occur. In typical case, seeing incongruent phonatory gesture can modify auditory percept phonetically (McGurk effects). In the present study, we used PET to identify brain activations during the presentation of inconsistent audiovisual phonetic signals.Results and Discussion1. Behavioral measures All the subjects answered /be/ when the acoustic /be/ was presented togethe … More r with visual /be/. On the other hand, nine of 12 subjects were answered /de/ (auditory illusion), when the acoustic /be/ was presented with visual /ge/ (inconsistent condition).2. Brain activationFor the nine subjects in whom auditory illusion were observed, PET analysis was conducted using the voxel based image subtraction. When the condition of A-V stimulation of /be/-/ge/ was compared with that of A-V stimulation of /be/(A)-/be/(V), or A-V stimulation of /be/-/VN/, activated areas were observed in the amvadala, the anteriorpart of the cingulate gyrus, and the caudate head (p>0.005 uncorrected).Sekiyama et al (1994) indicated that many of subjects (especially in Japanese subjects) experienced a kind of unpleasant or strange feeling when they see the inconsistent A-V combination in McGurk stimulus. The activation of regions in the limbic system observed in the presentation of inconsistent A-V combination (/be/-/ge/) might be related to this emotional reaction for the unusual stimulus that is hardly experienced. Less

人类大脑在感知外部信号的过程中有效地整合了来自多感觉模态的信息。这种多模态处理比单模态处理有利于更快和更准确地识别信息。在语音感知中，人们已经知道说话人面部的视觉信息可以有效地用于语音感知。在发声中看到一致的视觉信息（面部运动）可以提高在背景噪声中听到语音的感知能力。即使当呈现的视觉提示与听觉输入不一致时，也会发生上述视听意图。在典型的情况下，看到不一致的发音手势可以修改听觉的语音感受（麦格克效应）。在本研究中，我们使用PET来识别不一致的视听语音信号的呈现过程中的大脑激活。行为测量所有的受试者回答/是/时，声学/是/一起提出， ...更多信息 r与visual /be/。另一方面，当听觉/be/与视觉/ge/（不一致条件）同时出现时，12名被试中有9名回答/de/（听觉错觉）.脑激活对于观察到听觉错觉的9名受试者，使用基于体素的图像减影进行PET分析。当A-V刺激/be/-/ge/与A-V刺激/be/（A）-/be/（V）或A-V刺激/be/-/VN/比较时，在环回、扣带回前部、和尾状头（p>0.005未校正）。Sekiyama等（1994）指出，许多受试者（尤其是日本被试）在看到McGurk刺激中不一致的A-V组合时会产生一种不愉快或奇怪的感觉。在呈现不一致的A-V组合（/be/-/ge/）时观察到的边缘系统区域的激活可能与这种对很少经历的不寻常刺激的情绪反应有关。少