Pathways and mechanisms underlying the visual enhancement of hearing in challenging environments.

语言感知对社会交流和日常生活质量至关重要。然而，许多人在语音理解上有困难，特别是在语音信号被声学扭曲的情况下（例如，被背景噪声）。在这种具有挑战性的环境下，看到说话者面部或嘴唇运动的视觉信息可以大大提高语音的可理解性。然而，具体的大脑机制，使听力容易受到这种挑战，以及那些实现听觉的视觉促进仍然知之甚少。我们在这里使用自然声音的两种关键操作，增加背景噪音和自然语音节奏的变化，来研究声音的神经编码在哪里以及如何受到这些声学挑战的影响。然后，我们使用这些操作来研究在哪里以及如何通过翔实的同时视觉信息来恢复正确的声音编码。为此，我们将行为测试与高密度电神经成像（脑磁图）和先进的计算信号分析相结合，用于成年人类听众，因此采用系统方法研究感官知觉期间的大规模大脑激活。我们的具体假设和问题是基于最近的见解，证明了听觉感知神经活动的节奏模式的重要性。我们直接研究了人类大脑中特定的语音编码特征是如何受到语音信号退化的影响，以及它们是如何被视觉信息恢复的。听力障碍在英国是一种非常常见的残疾，也是全球社会的负担（世卫组织），这使得它成为支撑终身健康的生物科学的一个重要话题。在这里获得的知识对于我们对听力背后的生理过程的基本理解以及理解大脑如何将视觉和听觉信息融合在一起以获得能够承受挑战性感官环境的感知是很有兴趣的。在这里获得的见解也将增强我们对听觉理解受损的神经机制的理解，因此可以提供关键的知识，以促进听觉假肢装置的改进或患者使用它们的培训。最后，理解语音信号及其在大脑中的表现之间的相互作用具有社会经济利益，因为它对日常生活中无处不在的人机语音界面的设计很重要。

The perception of speech is of paramount importance for social communication and the quality of our daily life. Yet, many individuals have difficulties in speech comprehension, especially so under conditions where the speech signal is acoustically distorted (e.g. by background noise). Visual information from seeing the speakers face or lip movements can greatly enhance speech intelligibility under such challenging circumstances. Yet, the specific brain mechanisms that render hearing susceptible to such challenges and those that implement the visual facilitation of hearing remain poorly understood.We here use two critical manipulations of natural sounds, the addition of background noise and changes in natural speech rhythm, to study where and how along auditory pathways the neural encoding of sounds is affected by these acoustic challenges. We then use these manipulations to study where and how proper sound encoding is restored by informative simultaneous visual information. To this end we use a combination of behavioural tests with high-density electric neuroimaging (Magnetoencephalography) and advanced computational signal analysis in adult human listeners, hence taking a systems approach studying large-scale brain activations during sensory perception. Our specific hypothesis and questions are based on recent insights demonstrating the importance of rhythmic patterns of neural activity for auditory perception. We directly investigate how specific signatures of speech encoding in the human brain are affected by degradation of the speech signal and how they are restored by visual information. Hearing difficulties are a highly common disability in the UK and a global burden to society (WHO), making this an important topic for the biosciences underpinning live-long health. The knowledge gained here is of interest for our basic understanding of the physiological processes underlying hearing and to understand how the brain merges visual and acoustic information to achieve a percept that can withstand challenging sensory environments. The insights gained here will also enhance our understanding of the neural mechanisms responsible for impaired auditory comprehension and hence could provide critical knowledge to facilitate the improvement of auditory prosthetic devices or the training of patients in using them. Finally, understanding the interplay between speech signals and their representation in the brain is of socio-economic interest as it is important for the design of human-computer voice-based interfaces, which are becoming ubiquitous in everyday life.