Hand gestures as visual prosody: BOLD responses to audio-visual alignment are modulated by the communicative nature of the stimuli
Hand gestures as visual prosody: BOLD responses to audio-visual alignment are modulated by the communicative nature of the stimuli
复制标题
DOI：
10.1016/j.neuroimage.2016.02.018
复制
发表时间：
2016-05-15
期刊：
影响因子：
5.7
通讯作者：
Soto-Faraco, Salvador
中科院分区：
文献类型：
--
作者：
Biau, Emmanuel;Moris Fernandez, Luis;Soto-Faraco, Salvador
在公开演讲期间，演讲者用自发的手势（节拍）来伴随演讲，这些手势与演讲的韵律轮廓紧密同步。有人提出，语音和节拍手势源自共同的潜在语言过程，其中语音韵律和节拍都用于强调相关信息。我们假设，通过时间去同步打破节拍和韵律之间的一致性，将调节对语音手势整合敏感的大脑区域的活动。为此，我们测量了参与者观看讲话者使用节拍手势的自然对话时的大胆反应。为了识别专门参与处理具有交流意图的手势的大脑区域，根据具有与手势相同的节奏和空间属性的任意视觉线索来评估节拍同步性。我们的结果表明，与任意视觉-语音配对相比，左 MTG 和 IFG 对与节拍同步的语音特别敏感。我们的研究结果表明，听众赋予节拍视觉韵律功能，与语音韵律结构互补。我们得出的结论是，语音感知中节拍手势的强调功能是通过对说话者用手传达的交流意图敏感的专门大脑网络来实例化的。 (C) 2016 Elsevier Inc. 保留所有权利。
During public addresses, speakers accompany their discourse with spontaneous hand gestures (beats) that are tightly synchronized with the prosodic contour of the discourse. It has been proposed that speech and beat gestures originate from a common underlying linguistic process whereby both speech prosody and beats serve to emphasize relevant information. We hypothesized that breaking the consistency between beats and prosody by temporal desynchronization, would modulate activity of brain areas sensitive to speech-gesture integration. To this aim, we measured BOLD responses as participants watched a natural discourse where the speaker used beat gestures.In order to identify brain areas specifically involved in processing hand gestures with communicative intention, beat synchrony was evaluated against arbitrary visual cues bearing equivalent rhythmic and spatial properties as the gestures. Our results revealed that left MTG and IFG were specifically sensitive to speech synchronized with beats, compared to the arbitrary vision-speech pairing. Our results suggest that listeners confer beats a function of visual prosody, complementary to the prosodic structure of speech. We conclude that the emphasizing function of beat gestures in speech perception is instantiated through a specialized brain network sensitive to the communicative intent conveyed by a speaker with his/her hands. (C) 2016 Elsevier Inc. All rights reserved.