The connectivity signature of co-speech gesture integration: The superior temporal sulcus modulates connectivity between areas related to visual gesture and auditory speech processing
The connectivity signature of co-speech gesture integration: The superior temporal sulcus modulates connectivity between areas related to visual gesture and auditory speech processing
复制标题
共同语音手势整合的连接特征：颞上沟调节与视觉手势和听觉语音处理相关的区域之间的连接
DOI：
10.1016/j.neuroimage.2018.07.037
复制
发表时间：
2018
期刊：
影响因子：
5.7
通讯作者：
Yifei He
中科院分区：
文献类型：
--
作者：
B. Straube;A. Wroblewski;A. Jansen;Yifei He
人类整合了通过语言和手势交流的信息。功能磁共振成像(FMRI)研究表明，后颞叶上沟(STS)和邻近的脑回与多感觉整合有关。然而，代表这一基本组合过程的连通性模型仍然缺乏。在这里，我们使用fMRI的动态因果模型来分析分别与听觉语言、视觉手势相关和整合加工相关的颞中回(MTG)、枕叶皮质(OC)和短时脑区(STS)之间的有效连接模式，以揭示内在意义手势(例如，竖起大拇指手势)与对应言语整合的神经机制。20名参与者观看了参与者在执行内容判断任务时在德语或俄语句子语境中执行内在有意义手势，或在没有手势的情况下说德语句子的视频。此外，该模型还考虑了两种连接方式(OC-→-→-STS)的调制，以及STS对MTG-OC之间双向连接的非线性调制效应。枕叶通路的耦合强度(OC→ST)与任务绩效中的手势相关优势相关，而时间通路(MTG→ST)与纯语音条件下的绩效相关。MTG和OC之间的耦合与后续的手势-德语条件下句子的记忆成绩呈负相关。我们的模型为更好地理解网络层面的语音-手势整合提供了第一步。它证实了STS在视听整合过程中的重要性，表明该区域抑制了直接的听觉-视觉耦合。
Humans integrate information communicated by speech and gestures. Functional magnetic resonance imaging (fMRI) studies suggest that the posterior superior temporal sulcus (STS) and adjacent gyri are relevant for multisensory integration. However, a connectivity model representing this essential combinatory process is still missing. Here, we used dynamic causal modeling for fMRI to analyze the effective connectivity pattern between middle temporal gyrus (MTG), occipital cortex (OC) and STS associated with auditory verbal, visual gesture-related, and integrative processing, respectively, to unveil the neural mechanisms underlying integration of intrinsically meaningful gestures (e.g., “Thumbs-up gesture”) and corresponding speech.20 participants were presented videos of an actor either performing intrinsic meaningful gestures in the context of German or Russian sentences, or speaking a German sentence without gesture, while performing a content judgment task.The connectivity analyses resulted in a winning model that included bidirectional intrinsic connectivity between all areas. Furthermore, the model included modulations of both connections to the STS (OC→STS; MTG→STS), and non-linear modulatory effects of the STS on bidirectional connections between MTG and OC. Coupling strength in the occipital pathway (OC→STS) correlated with gesture related advantages in task performance, whereas the temporal pathway (MTG→STS) correlated with performance in the speech only condition. Coupling between MTG and OC correlated negatively with subsequent memory performance for sentences of the Gesture-German condition.Our model provides a first step towards a better understanding of speech-gesture integration on network level. It corroborates the importance of the STS during audio-visual integration by showing that this region inhibits direct auditory-visual coupling.
登录
查看更多内容
影响因子：
3.7
作者：
Pelphrey, KA;Morris, JP;McCarthy, G
通讯作者：
McCarthy, G
影响因子：
4.2
作者：
Corinna Pehrs;L. Deserno;J. Bakels;Lorna H. Schlochtermeier;Hermann Kappelhoff;A. Jacobs;Thomas Fritz;S. Koelsch;L. Kuchinke
通讯作者：
Corinna Pehrs;L. Deserno;J. Bakels;Lorna H. Schlochtermeier;Hermann Kappelhoff;A. Jacobs;Thomas Fritz;S. Koelsch;L. Kuchinke
影响因子：
2.5
作者：
Robins, Diana L.;Hunyadi, Elinora;Schultz, Robert T.
通讯作者：
Schultz, Robert T.
影响因子：
3.7
作者：
Johnson-Frey, SH;Newman-Norlund, R;Grafton, ST
通讯作者：
Grafton, ST
影响因子：
5.7
作者：
Müller VI;Cieslik EC;Turetsky BI;Eickhoff SB
通讯作者：
Eickhoff SB