Visual Form-Based Spoken Word Processing

描述(申请人提供)：言语可以在视觉的基础上学习和感知，依赖于看到口语的语言前期失聪的成年人就证明了这一点。视觉言语感知(唇读/朗读)并不局限于聋人，因为几乎每个人都表现出一些视觉口语识别；但能力因个人而异。为了解释唇读能力、它的个体差异、它的可塑性潜力以及它在斯彭语言处理系统中的作用，需要在神经水平上进行机械性的解释。基于广泛的知觉研究和初步的神经证据，这个开发项目将检验一个新的假设，即视觉口语表征存储在高级视觉皮质通路中。神经成像、计算和行为方法将被用来在听力和视力正常的人身上测试这一假设。要验证这一假设，需要克服传统功能磁共振成像(FMRI)数据分辨率方面的缺陷，并实现对可见语音刺激的控制。我们将应用先进的神经成像技术(快速适应功能磁共振成像和连通性分析)和语音差异的计算模型来定位大脑皮质活动，以响应可见的口头无意义音节和单词刺激。定位器将被用来在个体中识别高级视觉皮层中先前已被证明对可见语音具有选择性的区域、对于非语音面部运动具有选择性的区域、对于视觉正字法单词形式和语义处理具有选择性的区域、枕侧复合体、梭形脸部区域和人类视觉运动区域。这个项目的一个长期目标是确定视觉语音路径是否与听觉语音路径的组织方式相同，具有从语音特征到单词的多个层次的表征。视觉语音通路可能是层级较浅的，非特定于语音的视觉特征直接投射到高级视觉皮层区域的语音表征。这个项目将测试关于局部大脑区域对可见口语单词和音节的选择性的具体假设。它还将在全脑分析中调查适应性、连通性和心理生理学效应，分析重点是可能有助于语音视觉感知的大脑区域之间的关系。该项目涉及语音科学研究人员和视觉科学家之间的高度创新合作，使用多学科方法可以使这两个研究领域受益。详细了解复杂而动态的视觉言语刺激是如何在视觉通路中被处理的，将有助于视觉科学。临床相关性。如果获得了高级视觉系统参与语音识别的证据，那么视觉科学就可以用创新的临床方法来训练听力损失的人的唇读。听力损失的人在面对面的交流中经常依赖于可视语言，提高使用视觉信息的能力可以改善他们的生活质量。一个多世纪以来，人们一直在寻找唇读的个体差异的解释和改善唇读的方法。在未来的研究中，我们可以使用所提出的方法来隔离唇读差异的原因。研究可以检验这一假说，即不好的唇读对应于调节不佳的神经元口语视觉词汇表征，对不同和辨别不佳的刺激具有过度适应，而好的唇读对应于超过一定刺激不同程度的最小神经元适应。

DESCRIPTION (provided by applicant): Speech can be learned and perceived on the basis of vision, as demonstrated by prelingually deaf adults who rely on seeing spoken language. Visual speech perception (lipreading/speechreading) is not limited to deaf individuals, as almost everyone demonstrates some visual spoken word recognition; but ability varies widely from individual to individual. A mechanistic account at the neural level is needed to help explain lipreading ability, its individual variation, its potential for plasticity, and its role in the spoen language processing system. Based on extensive perceptual research and initial neural evidence, this development project will test a novel hypothesis that visual spoken word representations are stored in the high-level vision cortical pathway. Neuroimaging, computational, and behavioral methods will be used to test this hypothesis in individuals with normal hearing and vision. To test this hypothesis requires overcoming deficiencies in the resolution of conventional functional magnetic resonance imaging (fMRI) data and achieving control over visible spoken stimuli. We will apply advanced neuroimaging techniques (rapid adaptation fMRI and connectivity analyses) and computational modeling of speech dissimilarity to localize in the cerebral cortex activity in response to visible spoken nonsense syllable and word stimuli. Localizers will be used to identify in individuals an area in high-level visual corte that has previously been shown to be selective for visible speech, an area selective for non-speech face motion, areas selective for visual orthographic word forms and for semantic processing, the lateral occipital complex, the fusiform face area, and the human visual motion area. A longterm goal of this project is to determine whether the visual speech pathway is organized the same way as the auditory speech pathway, with multiple levels of representation from speech features to words. The visual speech pathway might be hierarchically shallow, with visual features that are not specific to speech projecting directly to speech representations in high-level vision cortical areas. This project will test specific hypotheses about the selectivity f localized brain areas for visible spoken words and syllables. It will also investigate adaptation, connectivity, and psychophysiological effects in whole brain analyses focused on relationships among brain areas that might contribute to visual perception of speech. The project involves a highly innovative collaboration between speech science researchers and vision scientists using a multidisciplinary approach that can benefit both areas of research. Detailed understanding of how the complex and dynamic visual speech stimulus is processed in the visual pathway will contribute to vision science. Clinical relevance. If evidence is obtained for high-level visual system involvement in speech recognition, innovative clinical methods for training lipreading in those with hearing loss become available from vision science. Individuals with hearing loss frequently depend on visible speech in face-to-face communication, and enhanced ability to use visual information could improve the quality of their lives. Explanations for individual difference in lipreading and methods to improve lipreading have been sought for over a century. In future studies, we can use the proposed methods to isolate causes for lipreading differences. Studies can test the hypothesis that poor lipreading corresponds to poorly tuned neuronal spoken visual word form representations with excessive adaptation for dissimilar and poorly discriminated stimuli, and good lipreading corresponds to minimal neuronal adaptation beyond a certain level of stimulus dissimilarity.