Multisensory Training for Unisensory Perceptual Learning

当听觉语音刺激由于外部因素(如噪声)或内部因素(如 听力损失，能够看到说话者，通常会改善言语感知。这种影响通常有如下解释 由于将听觉和视觉语音信息结合在一起，从而使更多的语音信息可用 对感知者来说。在这个项目中，我们将检验新的假设，即视觉信息也可以引导 对听觉言语刺激中信息的知觉学习。听觉言语感知被改变为 使用视听(AV)语音刺激的体验或训练的结果。我们假设这是一种 知觉学习效应是利用听觉和视觉之间的相关性或偶然性的能力 输入刺激中的语音提示。这些关系之所以存在，是因为语言的生物力学产生了 视觉和声音；接受者在演讲中获得关于这些视听关系的隐含知识 他们在日常生活中相遇。听觉语音知觉学习的实验将在NORMAL 听力和视力正常的成年人。刺激将来自自然录音，声学语音将是 通过声码进行降级。视频要么是自然的，要么是合成的。培训将使用配对助理任务 其中参与者将学习每个双音节无意义单词与其分配的 胡说八道的画面。学习的衡量标准将是训练期间的分数和纯听觉测试的分数。 这是在结对合作伙伴培训之后进行的。泛化的措施将是新的辅音识别 两个音节的无意义单词，在任何训练前和实验结束时。在实验中1、 同步AV语音实现最佳听觉学习的必要性将通过同步进行测试 以及不同步的语音、印刷文字和仅听觉的控制条件。在实验中2、视觉能力 将通过不匹配听觉和听觉来测试扭曲听觉语音感知表征的语音 训练过程中的视觉刺激。在实验中3、具有自然或合成视频刺激的AV条件将是 对比测试视觉语音信息的质量是否影响听觉知觉学习。在……里面 实验在视听训练中，通过与不同的说话者进行训练，将挑战对听觉语音提示的注意力 对于每个配对伙伴。临床相关性：知觉学习是成功使用感官的关键 假体，例如，人工耳蜗和助听器。单感觉条件下的知觉学习 可能受到仅根据所提供的信息获取新的刺激信息的必要性的限制 通过单感假肢。具有自然关联性或偶然性的多感官刺激 实时感官可以更有效地指导单感知觉学习。研究是 需要了解多感官学习如何实现这些目标，以及如何开展实践训练 政权。

When auditory speech stimuli are degraded due to external factors such as noise or internal factors such as hearing loss, being able to see the talker typically improves speech perception. This effect is usually explained as the result of auditory and visual speech information combining, so that more speech information is available to the perceiver. In this project, we will examine the novel hypothesis that the visual information can also guide perceptual learning of the information in the auditory speech stimulus. Auditory speech perception is altered as a result of experience or training with audiovisual (AV) speech stimuli. We hypothsize that the basis for the perceptual learning effect is the ability to exploit correlations or contingencies between auditory and visual speech cues in the input stimuli. These relationships exist because the biomechanics of speech produce both sights and sounds; and perceivers gain implicit knowledge of these audiovisual relationships in the speech they encounter in daily life. Experiments on auditory speech perceptual learning will be carried out with normal hearing and sighted adults. The stimuli will derive from natural recordings and the acoustic speech will be degraded by vocoding. The video will be either natural or synthetic. Training will use a paired-associates task in which participants will learn associations between each two-syllable nonsense word and its assigned nonsense picture. The measure of learning will be scores during training and scores in an auditory-only test that follows paired-associates training. The measure of generalization will be consonant identification in new two-syllable nonsense words, before any training and at the conclusion of the experiment. In Exp. 1, the necessity for synchronized AV speech to achieve optimal auditory learning will be tested with synchronized and desynchronized speech, printed words, and auditory-only control conditions. In Exp. 2, the ability of visual speech to distort auditory speech perceptual representations will be tested by mismatching the auditory and visual stimuli during training. In Exp. 3, AV conditions with natural or synthesized video stimuli will be compared to test whether the quality of the visual speech information affects auditory perceptual learning. In Exp. 4, attention to auditory speech cues during AV training will be challenged by training with different talkers for each paired-associate. Clinical relevance: Perceptual learning is critical to successful use of sensory prostheses, for example, cochlear implants and hearing aids. Perceptual learning under unisensory conditions can be limited by the necessity to access new stimulus information based only on the information delivered through the unisensory prosthesis. Multisensory stimuli with natural correlations or contingencies delivered to the senses in realtime could guide unisensory perceptual learning more effectively and efficiently. Research is needed to understand how multisensory learning achieves these goals, and how to develop practical training regimes.