Visual Speech Perception and Neural Processing

会说话的脸经常被描述为语音信息的自然但辅助来源 在面对面的谈话中。但对于有严重听力障碍的人来说，通常是听觉障碍 语音信息是辅助信息来源。幸运的是，许多语言前期失聪的成年人依赖视觉 用于交流的语音信息，表现出增强的视觉语音感知。一些听证 成年人也是有能力的视觉言语感受器。人们对视觉言语感知(唇语)知之甚少。 可视语音信号的一个重要特征是它传递的语音信息比 声学语音信号，但它提供了足够的语音信息来识别，很高比例的 词汇。我们之前的研究表明，视觉语音信息的知觉相似性结构 与视觉刺激的物理相似结构同构。我们假设专业的说谎者是 对视觉语音刺激中微小语音差异敏感的结果，以及专业的聋人唇读器 比听觉唇读器对感知地呈现音素的语音信息更敏感 不一样的。这一假设将在旨在利用视觉的辨别实验中得到验证。 自然发音音节的知觉相似性。然而，自然语音刺激是极其复杂的， 我们还将使用我们新的可视语音合成器来调查我们过去使用的物理数据 表征语音刺激空间的是自然信息的充分表示 视觉语音刺激。最后，我们的长期目标是确定大脑皮层层次结构中的位置和时间 视觉语音刺激被处理为其语音信息。为了实现这一长期目标，一个 电生理学研究将利用所学到的视觉语音差异来定位视觉语音 在大脑皮层进行时间和空间处理。与公共卫生的相关性：患有 听力损失随着人口老龄化而增加。视觉是语音输入的主要来源，只要是听觉的 知觉受损。了解增强视觉言语感知的机制可能会导致 更有效和高效的临床应用，如评估语音感知和提供 对听力受损的个人进行补救。我们的可视语音合成器将用于培训 聋儿词汇发展与唇读能力研究。

The talking face is very frequently characterized as a natural but ancillary source of speech information during face-to-face conversation. But typically for individuals with profound hearing impairments, auditory speech information is the ancillary source. Fortunately, many prelingually deaf adults, who rely on visual speech information for communication, demonstrate enhanced visual speech perception. Some hearing adults are also capable visual speech perceivers. Little is known about visual speech perception (lipreading). An important characteristic of the visual speech signal is that it conveys less phonetic information than does the acoustic speech signal, yet it affords adequate phonetic information to recognize, a high percentage of words.Our previous research suggests that the perceptual similarity structure of visual phonetic information is isomorphic with the physical similarity structure of visual stimuli. We hypothesize that expert lipreading is the result of sensitivity to small phonetic differences in visual speech stimuli, and that expert deaf lipreaders are more sensitive than hearing lipreaders to the phonetic information that renders phonemes perceptually dissimilar. This hypothesis will be tested in discrimination experiments designed to exploit the visual perceptual similarity of natural spoken syllables. However, natural speech stimuli are extremely complex, and we will also use our new visual speech synthesizer to investigate whether the physical data we used to characterize the speech stimulus space are an adequate representation of the information in the natural visual speech stimuli. Finally, our longterm goal is to determine where and when in the cortical hierarchy visual speech stimuli are processed for their phonetic information. To approach this longterm goal, an electrophysiology study will use what is learned about visual speech dissimilarity to localize visual phonetic processing temporally and spatially in cortex. Relevance to public health: Numbers of individuals with hearing loss increase as the population ages. Vision is a major source of speech input whenever auditory perception is impaired. Understanding the mechanisms that enhnace visual speech perception could lead to more effective and efficient clinical applications, such as evaluating speech perception and providing remediation to individuals with hearing impairment. Our visual speech synthesizer will be useful for training deaf children in vocabulary development and lipreading.