Audiovisual integration of emotional signals in voice and face: An event-related fMRI study
Audiovisual integration of emotional signals in voice and face: An event-related fMRI study
复制标题
DOI：
10.1016/j.neuroimage.2007.06.020
复制
发表时间：
2007-10-01
期刊：
影响因子：
5.7
通讯作者：
Wildgruber, Dirk
中科院分区：
文献类型：
--
作者：
Kreifelts, Benjamin;Ethofer, Thomas;Wildgruber, Dirk
在自然环境中，非语言情感交流是多通道的(即语音、旋律、面部表情)，涉及到情感表达的多样性。理解这些交流信号，并将它们整合成一种共同的认知，对于成功的社交行为至关重要。虽然以前的许多研究都集中在仅在听觉或视觉通道中进行情绪交流的神经生物学，但对听觉和视觉非语言情绪信息的多模式整合的了解要少得多。本研究使用事件相关功能磁共振对这一过程进行了研究。行为数据显示，与视觉和听觉刺激相比，非语言情感信息的视听呈现导致正确分类的刺激显著增加。当将视听与听觉和视觉条件进行对比时，这种行为增益与双侧后颞叶上回(PSTG)和右侧丘脑的激活增强平行。此外，这些大脑区域的一个特征，证实了它们在情绪整合过程中的作用，即在双峰条件下，分类准确率的提高与大胆反应的强度之间存在线性关系。此外，在视听刺激过程中，视听整合区域与相关的听觉和视觉皮质之间的有效连通性增强，为实现多模式整合的神经过程提供了进一步的洞察。最后，我们能够证明，与中性刺激相比，假设的整合部位对带有情绪性非语言内容的刺激具有更高的敏感性。(C)2007 Elsevier Inc.保留所有权利。
In a natural environment, non-verbal emotional communication is multimodal (i.e. speech melody, facial expression) and multifaceted concerning the variety of expressed emotions. Understanding these communicative signals and integrating them into a common percept is paramount to successful social behaviour. While many previous studies have focused on the neurobiology of emotional communication in the auditory or visual modality alone, far less is known about multimodal integration of auditory and visual non-verbal emotional information. The present study investigated this process using event-related fMRI. Behavioural data revealed that audiovisual presentation of non-verbal emotional information resulted in a significant increase in correctly classified stimuli when compared with visual and auditory stimulation. This behavioural gain was paralleled by enhanced activation in bilateral posterior superior temporal gyrus (pSTG) and right thalamus, when contrasting audiovisual to auditory and visual conditions. Further, a characteristic of these brain regions, substantiating their role in the emotional integration process, is a linear relationship between the gain in classification accuracy and the strength of the BOLD response during the bimodal condition. Additionally, enhanced effective connectivity between audiovisual integration areas and associative auditory and visual cortices was observed during audiovisual stimulation, offering further insight into the neural process accomplishing multimodal integration. Finally, we were able to document an enhanced sensitivity of the putative integration sites to stimuli with emotional non-verbal content as compared to neutral stimuli. (C) 2007 Elsevier Inc. All rights reserved.