VISUAL PROCESSES IN SPEECH PERCEPTION

拟议研究的长期目标是开发一种全面的 模型来解释视觉信息在言语感知中的作用。 语音阅读是人类信息处理的一种形式，需要 观察者从物体的运动中获取语言信息 说话者的脸。几乎所有人，特别是有听力的人 在听力困难的情况下，依靠视觉信息进行听力损失。这个 本研究的主要目的是：1)研究视觉注意 视觉语音感知过程中面部运动的特定区域 在演讲稿阅读能力上存在差异的个人；以及2)确定 如果面部运动的特定区域在以下情况下发生变化 演讲稿阅读能力有所提高。项目1：知觉任务的执行情况 视觉语音处理将指定人脸运动的参数 视觉言语刺激和研究演讲稿读者如何注意、选择和 使用脸部运动进行视觉语音感知。图像处理 将使用光流分析等技术来量化人脸 运动，并测试关于语音的视觉语音提示的假设 感知力。眼睛监控技术将被用来精确地 确定演讲稿阅读器关注的面部区域。比较 将在朗读演讲稿水平不同的成年人中进行。 受试者包括听力正常的人和听力损失的人 先天性、早发性或成年性发病。项目2：感知中的机会 提高演讲稿阅读准确性的能力将决定1) 练习过程中的刺激、反馈和任务可变性得到改善 保留和转移，以及2)改善改变模式的受试者 对包含视觉语音的面部区域的注意和选择 暗示。听力和听力相似的实验和对照受试者 康复历史将会被比较。有关的科学知识 视觉言语知觉将有助于感觉的发展 艾滋病、自动语音识别系统和基于研究的 增强语音感知的干预方案。

The long-term goal of the proposed research is to develop a comprehensive model to explain the role of visual information in speech perception. Speechreading, a form of human information processing, requires an observer to obtain linguistic information from the movements of a talker's face. Virtually all people, and especially people with hearing loss, rely on visual information in difficult listening situations. The major objectives of this research are: 1) to study visual attention to specific regions of face motion during visual speech perception for individuals who differ in speechreading proficiency; and 2) to determine if attention to specific regions of face motion changes when speechreading improves. Project 1: performance on perceptual tasks in visual speech processing will specify parameters of face motion for visual speech stimuli and study how speechreaders attend to, select and use face motion for visual speech perception. Image processing techniques, such as optic flow analysis, will be used to quantify face motion, and test hypotheses about visual phonetic cues for speech perception. Eye-monitoring techniques will be used to precisely determine facial regions attended to by the speechreader. Comparisons will be made among adults who differ in speechreading proficiency. Subjects include those with normal hearing and hearing loss that may have a congenital-, early-, or adult-onset. Project 2: Chances in perceptual capabilities for improved speechreading accuracy will determine 1) if stimulus, feedback, and task variability during practice improve retention and transfer, and 2) if subjects who improve change patterns of attention and selection for facial regions containing visual phonetic cues. Experimental and control subjects with similar hearing and rehabilitation histories will be compared. Scientific knowledge about visual speech perception will contribute to the development of sensory aids, automatic speech recognition systems, and research-based intervention protocols to augment speech perception.