The effect of seeing the interlocutor on auditory and visual speech production in noise
The effect of seeing the interlocutor on auditory and visual speech production in noise
复制标题
DOI：
10.1016/j.specom.2015.08.001
复制
发表时间：
2015-11-01
影响因子：
3.2
通讯作者：
Davis, Chris
中科院分区：
文献类型：
--
作者：
Fitzpatrick, Michael;Kim, Jeesun;Davis, Chris
说话者在嘈杂环境中改变自己的言语产生部分是一种反射，但也是一种有意的交际策略，以促进语音信号向对话者的传递。先前的研究表明，说话者可以适应噪音中产生的语音的听觉和视觉元素。目前的研究考察了说话者是否会根据他们的交际环境来调整他们的听觉和视觉言语产生策略。参与者在不同的安静和噪音条件下完成了一个互动交流游戏，有/没有看到他们的对话者。我们发现，当对话者可以看到对方时，谈话者言语修饰的能量明显低于他们看不到对方的情况。此外，谈话者在嘈杂的条件下增加了视觉语言产生的显著性（以嘴唇面积和嘴唇宽度来衡量）。在一组感知研究中，我们使用来自生产研究的语音材料作为刺激，也证明了在面对面和非视觉交际条件下，语音生产策略的转变与所产生的语音信号的听觉和听觉视觉可理解性的变化相对应。结果表明，说话者主动监测他们的环境，并能够采取适当的言语生产策略，在不利的条件下进行高效和有效的沟通。(C) 2015 Elsevier B.V.版权所有
Talkers modify their speech production in noisy environments partly as a reflex but also as an intentional communicative strategy to facilitate the transmission of the speech signal to the interlocutor. Previous studies have shown that talkers can adapt both auditory and visual elements of speech produced in noise. The current study examined whether talkers adapt their auditory and visual speech production strategy as a function of their communicative setting. Participants completed an interactive communication game in various quiet and in noise conditions with/without being able to see their interlocutor. We found that the energy of talkers' speech modifications was significantly lower in conditions when interlocutors could see each other relative to conditions where they could not. Further, talkers increased the saliency of their visual speech production (measured as lip-area and lip-width) in noisy conditions for face-to-face communication. In a set of perception studies, using the speech materials from the production study as stimuli, we also demonstrated that the shift in speech production strategy across the face-to-face and non-visual communicative conditions corresponded with changes in the auditory and auditory visual intelligibility of the speech signal produced. The results suggest that talkers actively monitor their environment and are able to adopt appropriate speech production strategies for efficient and effective communication in adverse conditions. (C) 2015 Elsevier B.V. All rights reserved.