Listening effort and audiovisual speech by L1 ("native") and LX ("nonnative") English speakers

项目总结 演讲文献中最有说服力的发现之一是，听众能够理解演讲 当他们既能看到也能听到说话者时，他们会更成功。然而，有相当大的 关于看到说话者的脸是增加还是减少认知和注意力的争论 成功处理语音的要求(“听力努力”)，相对于单纯的听力而言。 这项研究中相互矛盾的发现可能部分是由于缺乏对 缓和视觉输入对听力努力影响的多种因素。建议数 研究将包括本科生研究人员参与这一过程的每一步，将澄清 视听演讲如何影响听力努力的多个方面，并将评估是否和 这些发现是如何被演讲任务的难度所缓和的。此外，它还将评估 这些发现是否适用于历史上被排除在演讲之外的参与者 研究：英语不是他们的第一语言的人。 这些研究将会： 1)评估视听演讲对听力努力的影响，其量化方法如下 三种常用的努力测量方法：主观努力、双重任务成本和回忆。 2)评估信噪比对以下工作的调节效果 处理纯音频和视听语音。 3)评估听者的语言背景如何影响与以下内容相关的努力 处理纯音频和视听语音。 这将是第一项评估听力努力是否以及如何与 对于先于任何其他语言学习英语的人来说，视听语音的处理是不同的 以及那些在学习英语之前学习了另一门语言的人。鉴于视听的无处不在 说话和保持高水平的倾听努力会导致精神疲劳或 了解视觉信息如何影响听力努力具有重要的实际意义 以及临床后果。

Project summary One of the most robust findings in the speech literature is that listeners can understand speech more successfully when they can see as well as hear the talker. However, there is considerable debate about whether seeing a talker’s face increases or decreases the cognitive and attentional requirements for successfully processing speech (“listening effort”), relative to hearing alone. Conflicting findings in the research may be due, in part, to a lack of systematic evaluation of multiple factors that moderate the effects of visual input on listening effort. The proposed research, which will involve undergraduate researchers at every step of the process, will clarify how audiovisual speech affects multiple facets of listening effort and will assess whether and how these findings are moderated by the difficulty of the speech task. Further, it will assess whether these findings extend to participants who have historically been excluded from speech research: those for whom English is not their first language. The studies will: 1) Evaluate the consequences of audiovisual speech on listening effort, as quantified by three commonly-used measures of effort: subjective effort, dual-task costs, and recall. 2) Assess the moderating effect of signal-to-noise ratio on the effort associated with processing audio-only and audiovisual speech. 3) Evaluate how the language background of the listeners affects the effort associated with processing audio-only and audiovisual speech. This will be the first study to assess whether and how the listening effort associated with processing audiovisual speech differs for people who learned English before any other language and those who learned another language before English. Given the ubiquity of audiovisual speech and the fact that maintaining high levels of listening effort can lead to mental fatigue or distress, understanding how visual information affects listening effort has important practical and clinical consequences.