The effects of visual beats on prosodic prominence: Acoustic analyses, auditory perception and visual perception
The effects of visual beats on prosodic prominence: Acoustic analyses, auditory perception and visual perception
复制标题
DOI：
10.1016/j.jml.2007.06.005
复制
发表时间：
2007-10-01
影响因子：
4.3
通讯作者：
Swerts, Marc
中科院分区：
文献类型：
--
作者：
Krahmer, Emiel;Swerts, Marc
说话者使用声音提示（音高重音）来表明某个单词很重要，但也可以使用视觉提示（节拍姿势、点头、眉毛运动）来达到此目的。尽管这些声音和视觉线索是相关的，但这种关系的确切性质还远未得到很好的理解。我们研究产生视觉节拍是否会导致语音中声学突出性的实现方式发生变化，以及是否会导致观察者感知突出性的方式发生变化。对于实验一（“制作节拍”），我们使用原始的实验范例，其中指示说话者实现具有不同声学和视觉线索分布的目标句子以突出显示。声学分析表明，视觉节拍的产生确实对同时发生的语音的声学实现有影响，特别是对持续时间和较高共振峰（F-2 和 F-3）的影响，与视觉节拍的类型以及音调重音的存在和位置无关。在实验二（“听觉节拍”）中，发现视觉节拍对目标词的感知显着性有显着影响。当说话者做出节拍姿势、眉毛移动或点头时，伴随的单词会以相对更重的口语强度产生。最后，在实验三（“看到节拍”）中，我们发现，当参与者看到说话者意识到某个单词上有视觉节拍时，他们会认为它比没有看到节拍手势时更加突出。 (c) 2007 Elsevier Inc. 保留所有权利。
Speakers employ acoustic cues (pitch accents) to indicate that a word is important, but may also use visual cues (beat gestures, head nods, eyebrow movements) for this purpose. Even though these acoustic and visual cues are related, the exact nature of this relationship is far from well understood. We investigate whether producing a visual beat leads to changes in how acoustic prominence is realized in speech, and whether it leads to changes in how prominence is perceived by observers. For Experiment I ("making beats") we use an original experimental paradigm in which speakers are instructed to realize a target sentence with different distributions of acoustic and visual cues for prominence. Acoustic analyses reveal that the production of a visual beat indeed has an effect on the acoustic realization of the co-occurring speech, in particular on duration and the higher formants (F-2 and F-3), independent of the kind of visual beat and of the presence and position of pitch accents. In Experiment II ("hearing beats"), it is found that visual beats have a significant effect on the perceived prominence of the target words. When a speaker produces a beat gesture, an eyebrow movement or a head nod, the accompanying word is produced with relatively more spoken emphasis. In Experiment III ("seeing beats"), finally, it is found that when participants see a speaker realize a visual beat on a word, they perceive it as more prominent than when they do not see the beat gesture. (c) 2007 Elsevier Inc. All rights reserved.