The N400 and late occipital positivity in processing dynamic facial expressions with natural emotional voice
The N400 and late occipital positivity in processing dynamic facial expressions with natural emotional voice
复制标题
DOI：
10.1097/wnr.0000000000001669
复制
发表时间：
2021-07-07
期刊：
影响因子：
1.7
通讯作者：
Arao, Hiroshi
中科院分区：
文献类型：
--
作者：
Mori, Kazuma;Tanaka, Akihiro;Arao, Hiroshi
人们需要多通道的情感互动才能生活在社会环境中。一些使用动态面部表情和情绪声音的研究报告称，多通道情绪不协调会唤起事件相关电位(ERPs)的早期感觉成分，而另一些研究则发现了晚期认知成分。两种不同结果的整合机制尚不清楚。我们推测，正是多通道集成框架中的语义分析调用了后期的ERP组件。使用情绪一致或不一致的动态面孔和自然声音进行电生理实验，以促进语义分析。为了研究事件相关电位成分的自上而下的调节，通过两个任务来操纵注意力，这两个任务引导参与者注意面部表情和声音表情。我们的结果揭示了面部和声音情绪表达之间的相互作用，表现为听觉N400事件相关电位幅度的调制，而不是N1和P2幅度的调制，仅在面孔注意任务中不协调的情绪面声组合。只有在注意发声的任务中，才会出现枕部晚期的正电位幅度。总体而言，这些发现支持语义分析是唤起晚认知成分的关键因素的观点。对这些事件相关事件的任务效应表明，自上而下的注意不仅改变了事件相关电位的幅度，而且改变了事件相关事件本身的成分。我们的结果暗示了大脑中情绪面孔-声音处理的原理，这可能是日常交流中复杂的视听互动的基础。
People require multimodal emotional interactions to live in a social environment. Several studies using dynamic facial expressions and emotional voices have reported that multimodal emotional incongruency evokes an early sensory component of event-related potentials (ERPs), while others have found a late cognitive component. The integration mechanism of two different results remains unclear. We speculate that it is semantic analysis in a multimodal integration framework that evokes the late ERP component. An electrophysiological experiment was conducted using emotionally congruent or incongruent dynamic faces and natural voices to promote semantic analysis. To investigate the top-down modulation of the ERP component, attention was manipulated via two tasks that directed participants to attend to facial versus vocal expressions. Our results revealed interactions between facial and vocal emotional expressions, manifested as modulations of the auditory N400 ERP amplitudes but not N1 and P2 amplitudes, for incongruent emotional face-voice combinations only in the face-attentive task. A late occipital positive potential amplitude emerged only during the voice-attentive task. Overall, these findings support the idea that semantic analysis is a key factor in evoking the late cognitive component. The task effect for these ERPs suggests that top-down attention alters not only the amplitude of ERP but also the ERP component per se. Our results implicate a principle of emotional face-voice processing in the brain that may underlie complex audiovisual interactions in everyday communication.