Pathways and mechanisms underlying the visual enhancement of hearing in challenging environments.

在充满挑战的环境中视觉增强听力的途径和机制。

• 批准号: BB/L027534/1
• 负责人: Christoph Kayser
• 金额: $ 47.12万
• 依托单位: University of Glasgow
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2014
• 资助国家: 英国
• 起止时间: 2014 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=BB%2FL027534%2F1
• 关键词: Pathways; mechanisms; underlying; visual; enhancement

The perception of speech is of paramount importance for social communication and the quality of our daily life. Yet, many individuals have difficulties in speech comprehension, especially so under conditions where the speech signal is acoustically distorted (e.g. by background noise). Visual information from seeing the speakers face or lip movements can greatly enhance speech intelligibility under such challenging circumstances. Yet, the specific brain mechanisms that render hearing susceptible to such challenges and those that implement the visual facilitation of hearing remain poorly understood.We here use two critical manipulations of natural sounds, the addition of background noise and changes in natural speech rhythm, to study where and how along auditory pathways the neural encoding of sounds is affected by these acoustic challenges. We then use these manipulations to study where and how proper sound encoding is restored by informative simultaneous visual information. To this end we use a combination of behavioural tests with high-density electric neuroimaging (Magnetoencephalography) and advanced computational signal analysis in adult human listeners, hence taking a systems approach studying large-scale brain activations during sensory perception. Our specific hypothesis and questions are based on recent insights demonstrating the importance of rhythmic patterns of neural activity for auditory perception. We directly investigate how specific signatures of speech encoding in the human brain are affected by degradation of the speech signal and how they are restored by visual information. Hearing difficulties are a highly common disability in the UK and a global burden to society (WHO), making this an important topic for the biosciences underpinning live-long health. The knowledge gained here is of interest for our basic understanding of the physiological processes underlying hearing and to understand how the brain merges visual and acoustic information to achieve a percept that can withstand challenging sensory environments. The insights gained here will also enhance our understanding of the neural mechanisms responsible for impaired auditory comprehension and hence could provide critical knowledge to facilitate the improvement of auditory prosthetic devices or the training of patients in using them. Finally, understanding the interplay between speech signals and their representation in the brain is of socio-economic interest as it is important for the design of human-computer voice-based interfaces, which are becoming ubiquitous in everyday life.

言语感知对于社会交流和我们日常生活的质量至关重要。然而，许多人在语音理解方面有困难，特别是在语音信号在声学上失真（例如，被背景噪声）的条件下。在这种具有挑战性的情况下，通过看到说话者的面部或嘴唇运动而获得的视觉信息可以大大提高语音的可懂度。然而，具体的大脑机制，使听力容易受到这样的挑战和那些实现听觉视觉促进仍然知之甚少，我们在这里使用两个关键的自然声音，添加背景噪音和自然语音节奏的变化，研究在哪里和如何沿着听觉通路的神经编码的声音是受这些声学的挑战。然后，我们使用这些操作来研究在哪里以及如何正确的声音编码恢复信息丰富的同时视觉信息。为此，我们使用高密度电神经成像（脑磁图）和先进的计算信号分析在成年人的听众的行为测试相结合，因此采取系统的方法研究大规模的大脑激活在感官知觉。我们的具体假设和问题是基于最近的见解，表明神经活动的节奏模式对听觉感知的重要性。我们直接研究了人类大脑中语音编码的特定特征如何受到语音信号退化的影响，以及它们如何被视觉信息恢复。听力障碍在英国是一种非常常见的残疾，也是一种全球性的社会负担（世卫组织），使其成为支撑终身健康的生物科学的一个重要课题。这里获得的知识对于我们基本了解听力相关的生理过程以及了解大脑如何融合视觉和声学信息以实现能够承受具有挑战性的感官环境的感知很有意义。在这里获得的见解也将提高我们对听觉理解受损的神经机制的理解，因此可以提供关键的知识，以促进改善听觉假体设备或培训患者使用它们。最后，理解语音信号及其在大脑中的表示之间的相互作用具有社会经济意义，因为它对于设计在日常生活中变得无处不在的基于语音的人机界面非常重要。

项目成果

Neural correlates of multisensory reliability and perceptual weights emerge at early latencies during audio-visual integration.

• DOI: 10.1111/ejn.13724
• 发表时间: 2017-11
• 期刊: The European journal of neuroscience
• 作者: Boyle SC;Kayser SJ;Kayser C
• 通讯作者: Kayser C

Multisensory causal inference in the brain.

• DOI: 10.1371/journal.pbio.1002075
• 发表时间: 2015-02
• 期刊: PLoS biology
• 影响因子: 9.8
• 作者: Kayser C;Shams L
• 通讯作者: Shams L

Neurocomputational mechanisms underlying cross-modal associations and their influence on perceptual decisions.

• DOI: 10.1016/j.neuroimage.2021.118841
• 发表时间: 2022-02-15
• 期刊: NEUROIMAGE
• 影响因子: 5.7
• 作者: Bolam, Joshua;Boyle, Stephanie C.;Ince, Robin A. A.;Delis, Ioannis
• 通讯作者: Delis, Ioannis

Conference Abstract A novel mutual information estimator for analysing M/EEG data and quantifying representational interactions between signals

会议摘要：一种新颖的互信息估计器，用于分析 M/EEG 数据并量化信号之间的代表性相互作用

• 发表时间: 2016
• 作者: Ince

Conference Abstract: Two opposite mechanisms for tracking speech in noise in the right frontal cortex in the presence or absence of visual speech information.

会议摘要：在存在或不存在视觉语音信息的情况下，在右额叶皮层的噪声中跟踪语音的两种相反机制。

• 发表时间: 2016
• 作者: Giordano B