Multisensory Training for Unisensory Perceptual Learning

当听觉语音刺激由于外部因素如噪声或内部因素如 听力损失，能够看到说话者通常改善言语感知。这种影响通常被解释为 作为听觉和视觉语音信息相结合的结果， 给感知者。在这个项目中，我们将检验一个新的假设，即视觉信息也可以引导 听觉言语刺激中信息的知觉学习。听觉言语感知被改变， 视听（AV）语音刺激的经验或训练的结果。我们假设， 知觉学习效应是利用听觉和视觉之间的相关性或偶然性的能力 输入刺激中的语音提示。这些关系的存在是因为言语的生物力学产生了 视觉和声音;感知者在言语中获得这些视听关系的隐含知识 他们在日常生活中遇到的。听觉言语知觉学习实验将在正常的 有听力和视力的成年人。刺激将来自自然录音，并且声学语音将是 被语音编码降级。视频将是自然的或合成的。培训将使用配对的同事任务 在其中，参与者将学习每个双音节无意义单词与其指定单词之间的联系 无意义的画面学习的衡量标准将是培训期间的分数和仅在知识测试中的分数 在配对律师培训之后推广的措施将是辅音识别在新的 两个音节的无意义的单词，在任何训练之前和实验结束时。在Exp. 1号 同步AV语音实现最佳听觉学习的必要性将用同步AV语音测试。 和去语音化的语音、印刷的文字和仅限语音的控制条件。在Exp. 2、视觉的能力 失真听觉语音感知表示的语音将通过不匹配听觉和 训练中的视觉刺激。在Exp. 3、AV条件与自然或合成的视频刺激将 比较测试视觉语音信息的质量是否影响听觉感知学习。在 Exp. 4，在AV训练期间对听觉言语线索的注意力将受到与不同谈话者一起训练的挑战 每一个配对的伙伴。临床相关性：感知学习对于成功使用感觉神经元至关重要。 假体，例如，耳蜗植入物和助听器。单一感知条件下的感知学习 可能受到仅基于所传递的信息来访问新刺激信息的必要性的限制 通过unisensory假肢。多感官刺激与自然的相关性或偶然性交付给 实时的感觉可以更有效地引导单一感知学习。研究是 需要了解多感官学习如何实现这些目标，以及如何开发实用的培训 政权。

When auditory speech stimuli are degraded due to external factors such as noise or internal factors such as hearing loss, being able to see the talker typically improves speech perception. This effect is usually explained as the result of auditory and visual speech information combining, so that more speech information is available to the perceiver. In this project, we will examine the novel hypothesis that the visual information can also guide perceptual learning of the information in the auditory speech stimulus. Auditory speech perception is altered as a result of experience or training with audiovisual (AV) speech stimuli. We hypothsize that the basis for the perceptual learning effect is the ability to exploit correlations or contingencies between auditory and visual speech cues in the input stimuli. These relationships exist because the biomechanics of speech produce both sights and sounds; and perceivers gain implicit knowledge of these audiovisual relationships in the speech they encounter in daily life. Experiments on auditory speech perceptual learning will be carried out with normal hearing and sighted adults. The stimuli will derive from natural recordings and the acoustic speech will be degraded by vocoding. The video will be either natural or synthetic. Training will use a paired-associates task in which participants will learn associations between each two-syllable nonsense word and its assigned nonsense picture. The measure of learning will be scores during training and scores in an auditory-only test that follows paired-associates training. The measure of generalization will be consonant identification in new two-syllable nonsense words, before any training and at the conclusion of the experiment. In Exp. 1, the necessity for synchronized AV speech to achieve optimal auditory learning will be tested with synchronized and desynchronized speech, printed words, and auditory-only control conditions. In Exp. 2, the ability of visual speech to distort auditory speech perceptual representations will be tested by mismatching the auditory and visual stimuli during training. In Exp. 3, AV conditions with natural or synthesized video stimuli will be compared to test whether the quality of the visual speech information affects auditory perceptual learning. In Exp. 4, attention to auditory speech cues during AV training will be challenged by training with different talkers for each paired-associate. Clinical relevance: Perceptual learning is critical to successful use of sensory prostheses, for example, cochlear implants and hearing aids. Perceptual learning under unisensory conditions can be limited by the necessity to access new stimulus information based only on the information delivered through the unisensory prosthesis. Multisensory stimuli with natural correlations or contingencies delivered to the senses in realtime could guide unisensory perceptual learning more effectively and efficiently. Research is needed to understand how multisensory learning achieves these goals, and how to develop practical training regimes.