Examining attention, memory performance, and listening effort (exAMPLE): Understanding listeners' cognitive performances in complex audiovisual communication settings with embodied conversational agents

检查注意力、记忆表现和听力努力(示例):通过具体对话代理了解听众在复杂视听通信环境中的认知表现

基本信息

项目摘要

Face-to-face communication is the most prevalent form of verbal information exchange. However, in complex and dynamic environments such as open-plan offices and busy bars, listeners of multi-talker conversations encounter unique challenges due to the acoustically complex, stimulus-rich, and dynamic nature of these multisensory settings. The effects of complex audiovisual presentations on comprehension and memory of multi-talker conversations, as well as the influence of note-taking while listening, remain largely unexplored. This research proposal aims to investigate audiovisual variations in realistic listening settings in Virtual Reality (VR) and assess their impact on listener attention, speech intelligibility, listening comprehension, as well as memory for conversational content and other talker-related aspects (e.g., who said what). Building upon our predecessor project, this study advances from simplified to intricate audiovisual immersive environments that are rich and vibrant in both auditory and visual stimuli. The scope of cognitive performance assessment expands from understanding and recalling conversational facts to encompass guided note-taking and the retrieval of additional conversational aspects, including attention allocation and guidance. To achieve this, we will develop immersive and interactive audiovisual virtual environments in which participants engage in conversations with two or more talkers, represented as embodied conversational agents. Subsequently, participants will be tested on their recall of conversational content and other conversation-related details. We will extend and adapt the Heard Text Recall (HTR) paradigm from our initial project to include note-taking (NT) and questions about talker-related (TR) aspects, finally resulting in the HTR-NT and HTR-TR tasks, respectively. By exploring the effects of audiovisual VR environment characteristics on the aforementioned cognitive performances, this research will provide guidelines for creating realistic and immersive verbal social interactions, thus facilitating ongoing VR-based cognitive research.
面对面交流是最普遍的口头信息交流形式。然而,在复杂和动态的环境中,如开放式办公室和忙碌的酒吧,听众的多说话者的谈话遇到独特的挑战,由于声学复杂,刺激丰富,和动态的性质,这些多感官设置。复杂的视听呈现对多人谈话的理解和记忆的影响,以及在听的时候记笔记的影响,在很大程度上还没有被探索。这项研究计划旨在调查虚拟现实(VR)中真实听力环境中的视听变化,并评估其对听众注意力,语音清晰度,听力理解以及对话内容和其他谈话者相关方面的记忆的影响(例如,谁说的)。这项研究以我们的前一个项目为基础,从简单的视听沉浸式环境发展到复杂的视听沉浸式环境,这些环境在听觉和视觉刺激方面都丰富而充满活力。认知表现评估的范围从理解和回忆会话事实扩展到包括指导性笔记和检索其他会话方面,包括注意力分配和指导。为了实现这一目标,我们将开发沉浸式和交互式视听虚拟环境,参与者与两个或更多的谈话者进行对话,代表具体的会话代理。随后,将测试参与者对对话内容和其他对话相关细节的回忆。我们将从我们最初的项目中扩展和调整听到的文本回忆(HTR)范式,以包括笔记(NT)和关于谈话者相关(TR)方面的问题,最终分别导致HTR-NT和HTR-TR任务。通过探索视听VR环境特征对上述认知表现的影响,本研究将为创建逼真和沉浸式的言语社交互动提供指导,从而促进正在进行的基于VR的认知研究。

项目成果

期刊论文数量(0)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Professorin Dr.-Ing. Janina Fels其他文献

Professorin Dr.-Ing. Janina Fels的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Professorin Dr.-Ing. Janina Fels', 18)}}的其他基金

Differential mechanisms of cognitive impairment due to task-irrelevant sounds in children and adults
儿童和成人与任务无关的声音导致认知障碍的不同机制
  • 批准号:
    401278266
  • 财政年份:
    2018
  • 资助金额:
    --
  • 项目类别:
    Research Grants
Individual Binaural Synthesis of Virtual Acoustic Scenes
虚拟声学场景的个体双耳合成
  • 批准号:
    402811912
  • 财政年份:
    2018
  • 资助金额:
    --
  • 项目类别:
    Research Grants
Auditory perception of sound reflections and source localization in dynamic scenes
动态场景中声音反射的听觉感知和声源定位
  • 批准号:
    249703531
  • 财政年份:
    2014
  • 资助金额:
    --
  • 项目类别:
    Research Grants
Intentional switching of auditory selective attention: Studies on dynamic binaural hearing in complex acoustic environments
听觉选择性注意的有意切换:复杂声环境下动态双耳听力研究
  • 批准号:
    198440333
  • 财政年份:
    2011
  • 资助金额:
    --
  • 项目类别:
    Research Grants
Age-dependent changes in mechanisms of auditory attentional shifts in selective hearing: Correlational approach to differences in auditory thresholds and experimental manipulation by hearing loss simulation
选择性听力中听觉注意力转移机制的年龄依赖性变化:听觉阈差异的相关方法和听力损失模拟的实验操作
  • 批准号:
    513033051
  • 财政年份:
  • 资助金额:
    --
  • 项目类别:
    Research Grants
Coordination Funds
协调基金
  • 批准号:
    444532506
  • 财政年份:
  • 资助金额:
    --
  • 项目类别:
    Priority Programmes
Evaluating auditory cognition in classroom environments across age groups from preschool children to adults using audiovisual virtual reality - EArAge-VR
使用视听虚拟现实 - EArAge-VR 评估从学龄前儿童到成人各年龄段课堂环境中的听觉认知
  • 批准号:
    444697733
  • 财政年份:
  • 资助金额:
    --
  • 项目类别:
    Priority Programmes

相似国自然基金

多模态超声VisTran-Attention网络评估早期子宫颈癌保留生育功能手术可行性
  • 批准号:
  • 批准年份:
    2022
  • 资助金额:
    30 万元
  • 项目类别:
    青年科学基金项目
Ultrasomics-Attention孪生网络早期精准评估肝内胆管癌免疫治疗的研究
  • 批准号:
  • 批准年份:
    2022
  • 资助金额:
    52 万元
  • 项目类别:
    面上项目
基于注意力的情感脑机接口研究与示范应用
  • 批准号:
    61075111
  • 批准年份:
    2010
  • 资助金额:
    10.0 万元
  • 项目类别:
    面上项目

相似海外基金

Examining Sleep, Circadian Rhythms, and Cognitive Functioning in Older Adults at Risk of Dementia
检查有痴呆风险的老年人的睡眠、昼夜节律和认知功能
  • 批准号:
    10606971
  • 财政年份:
    2023
  • 资助金额:
    --
  • 项目类别:
Re-examining links between screen time, health behaviors, and executive functioning: Validating an objective measure of screen exposure in a sample of young children
重新审视屏幕时间、健康行为和执行功能之间的联系:验证幼儿样本中屏幕暴露时间的客观测量
  • 批准号:
    10725847
  • 财政年份:
    2023
  • 资助金额:
    --
  • 项目类别:
Examining the Relationship between Executive Functioning and Cognitive Reappraisal Ability in Daily Life
检验日常生活中执行功能与认知重评能力之间的关系
  • 批准号:
    10654085
  • 财政年份:
    2023
  • 资助金额:
    --
  • 项目类别:
Examining the contributions of placebo effects in cognitive training
检查安慰剂效应在认知训练中的贡献
  • 批准号:
    10390772
  • 财政年份:
    2022
  • 资助金额:
    --
  • 项目类别:
Examining the contributions of placebo effects in cognitive training
检查安慰剂效应在认知训练中的贡献
  • 批准号:
    10630072
  • 财政年份:
    2022
  • 资助金额:
    --
  • 项目类别:
Examining Neurocircuit and Behavioral Effects in a Developmental Model for Indirect Pathway Hypofunction
检查间接通路功能障碍发育模型中的神经回路和行为效应
  • 批准号:
    10687115
  • 财政年份:
    2022
  • 资助金额:
    --
  • 项目类别:
Using everyday experience to orient attention in real-time: Examining the neural correlates of memory-guided attention in hearing
利用日常经验实时引导注意力:检查听力中记忆引导注意力的神经相关性
  • 批准号:
    547112-2020
  • 财政年份:
    2022
  • 资助金额:
    --
  • 项目类别:
    Alexander Graham Bell Canada Graduate Scholarships - Doctoral
Examining Neurocircuit and Behavioral Effects in a Developmental Model for Indirect Pathway Hypofunction
检查间接通路功能障碍发育模型中的神经回路和行为效应
  • 批准号:
    10449771
  • 财政年份:
    2022
  • 资助金额:
    --
  • 项目类别:
Using everyday experience to orient attention in real-time: Examining the neural correlates of memory-guided attention in hearing
利用日常经验实时引导注意力:检查听力中记忆引导注意力的神经相关性
  • 批准号:
    547112-2020
  • 财政年份:
    2021
  • 资助金额:
    --
  • 项目类别:
    Alexander Graham Bell Canada Graduate Scholarships - Doctoral
A randomized controlled trial examining the impact of a brief attention-based neurobehavioral transdiagnostic intervention on acute fear response
一项随机对照试验,研究基于短暂注意力的神经行为跨诊断干预对急性恐惧反应的影响
  • 批准号:
    10291622
  • 财政年份:
    2021
  • 资助金额:
    --
  • 项目类别:
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了