Evaluating the listening effort associated with audiovisual speech

评估与视听语音相关的听力努力

• 批准号: 9812559
• 负责人: Julia F Strand
• 金额: $ 40.46万
• 依托单位: CARLETON COLLEGE
• 依托单位国家: 美国
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/9812559
• 关键词: Affect; Aging; Algorithms; Attention; Auditory; Clinical; Cognitive; Conflict (Psychology); Cues; Device Designs; Distress; Elderly; Evaluation; Evaluation Research; Face; Future; Goals; Hearing; Hearing Aids; Impairment; Intervention; Language; Lead; Literature; Loudness; Masks; Measurement; Measures; Memory; Mental Fatigue; Methods; Nature; Noise; Participant; Population; Process; Reaction Time; Reporting; Research; Research Personnel; Resources; Signal Transduction; Source; Speech; Speed; Stimulus; Task Performances; Testing; Thinking; Vision; Visual; Work; audiovisual speech; base; clinical application; design; hard of hearing; hearing impairment; improved; information processing; listening comprehension; novel; speech in noise; speech processing; speech recognition; therapy design; undergraduate student; visual information; visual speech; visual stimulus

Project summary One of the most robust findings in the spoken word recognition literature is that listeners can understand speech more successfully when they can see as well as hear the talker. However, there is considerable debate about whether seeing a talker’s face increases or decreases the cognitive and attentional requirements for successfully processing speech (“listening effort”; LE), relative to hearing alone. Conflicting findings in the research may be due, in part, to a lack of systematic evaluation of multiple factors that moderate the effects of visual input on LE. The proposed research, which will involve undergraduate researchers at every step of the process, will be the largest-scale evaluation of this research question to date and will clarify how visual speech information affects LE. The studies will: 1. Compare the LE consequences of adding visual information from a talking face vs an abstract visual stimulus that provides timing but not phonetic information about speech. 2. Assess convergent validity of two commonly used LE measures (a speeded reaction time measure and a memory measure). 3. Determine how difficulty of listening condition (i.e., changes in the level of background noise) affects the LE associated with visual input. 4. Evaluate whether the relationship between visual information and LE differs as a function of the nature of the masking noise. In addition, the work will test whether visual speech information can reduce LE during a more naturalistic listening comprehension task than is typically used in the literature. Given the ubiquity of audiovisual speech and the fact that maintaining high levels of LE can lead to mental fatigue or distress, understanding how visual information affects LE has important practical and clinical consequences. The results of this study will inform the design of devices that generate visual stimuli to accompany spoken language in difficult listening situations, which may be particularly useful for the elderly or hard of hearing.

项目总结