Audiovisual integration of speech in noise: role of behavioural and neural mechanism

交流经常发生在噪音中。当音频信号减弱时，说话者面部和嘴部运动的视觉线索有助于改善我们的语言感知，尤其是对那些听力损失的人来说。然而，支持语音在噪声中的视听整合的神经机制在很大程度上是未知的。特别是，听觉皮层附近的区域与有或无听力损失的个体在噪音中语音的视听整合之间的因果关系尚未确定。目前的项目旨在解决这一差距，并澄清在噪音中支持视听语音的行为和神经机制。在实验1中，参与者在听视听演讲时，用眼动仪监测他们的目光。我们的目标是建立视觉注意、个体认知能力和言语感知之间的关系。实验2将有类似的设计，除了参与者在接受经颅磁刺激（TMS）后对听觉皮层附近的区域重复任务，以揭示支持的神经机制。实验3重复实验1和2，采用经颅直流电刺激（tDCS），通过机器学习技术量化视觉注意、个体认知能力和tDCS响应之间的相关性。

Communication often occurs in noise. When the audio signal is degraded, visual cues from the speaker's facial and mouth movements help to improve our speech perception, especially for those with hearing loss. However, the neural mechanisms supporting audiovisual integration of speech in noise are largely unknown. In particular, the causal correlation between areas near the auditory cortex and audiovisual integration of speech in noise for individuals with and without hearing loss is not yet established. The current project aims to address this gap and clarify the behavioural and neural mechanisms supporting audiovisual speech in noise. In Experiment 1, participants' gaze will be monitored with an eye-tracker when listening to audiovisual speech. We aim to establish the relationship between visual attention, individual cognitive abilities, and speech perception. Experiment 2 will have a similar design, except participants will repeat the task after receiving Transcranial Magnetic Stimulation (TMS) to an area nearby the auditory cortex to uncover the supporting neural mechanisms. Experiment 3 repeats experiment 1 and 2 with transcranial Direct Current Stimulation (tDCS) to quantify the correlation between visual attention, individual cognitive abilities, and tDCS responsiveness with machine learning techniques.