Visual Form-Based Spoken Word Processing

描述（由申请人提供）：语言可以在视觉的基础上学习和感知，正如语言前聋的成年人依靠视觉口语所证明的那样。视觉言语感知（唇读/言语阅读）并不局限于聋人，因为几乎每个人都表现出一定的视觉口语识别能力；但是能力因人而异。我们需要一个神经层面的机制来帮助解释唇读能力、它的个体差异、它的可塑性潜力以及它在口语处理系统中的作用。基于广泛的知觉研究和初步的神经证据，该开发项目将测试一个新的假设，即视觉口语表征存储在高级视觉皮层通路中。神经成像、计算和行为方法将用于在听力和视力正常的个体中验证这一假设。为了验证这一假设，需要克服传统功能磁共振成像（fMRI）数据分辨率的缺陷，并实现对可见言语刺激的控制。我们将应用先进的神经成像技术（快速适应功能磁共振成像和连通性分析）和语言差异的计算建模来定位大脑皮层活动对可见的口语无意义音节和单词刺激的反应。定位器将用于识别个体高级视觉皮质中一个区域，该区域先前被证明是对可见语言的选择性区域，对非语言面部运动的选择性区域，对视觉正字法单词形式和语义处理的选择性区域，侧枕复合体，梭状回面部区域和人类视觉运动区域。该项目的一个长期目标是确定视觉语音通路是否与听觉语音通路的组织方式相同，具有从语音特征到单词的多级表示。视觉语言通路可能层次较浅，与语言无关的视觉特征直接投射到高级视觉皮质区域的语言表征。这个项目将测试大脑局部区域对可见口语单词和音节的选择性的具体假设。它还将研究全脑分析中的适应性、连通性和心理生理效应，重点关注大脑区域之间的关系，这些区域可能有助于语言的视觉感知。该项目涉及语音科学研究人员和视觉科学家之间的高度创新合作，使用多学科方法，可以使两个领域的研究都受益。详细了解复杂动态的视觉言语刺激如何在视觉通路中被处理，将有助于视觉科学的发展。临床相关性。如果有证据表明高级视觉系统参与语音识别，那么从视觉科学中可以获得训练听力损失患者读唇的创新临床方法。听力损失的个体在面对面交流中经常依赖于视觉语言，增强使用视觉信息的能力可以改善他们的生活质量。一个多世纪以来，人们一直在寻找唇读中个体差异的解释和提高唇读的方法。在未来的研究中，我们可以使用所提出的方法来分离唇读差异的原因。研究可以验证这样的假设，即唇读能力差对应于神经元口语视觉词的形式表征不佳，对不同和区分不佳的刺激有过度的适应，而唇读能力好对应于超过一定程度的刺激不相似的最小神经元适应。

DESCRIPTION (provided by applicant): Speech can be learned and perceived on the basis of vision, as demonstrated by prelingually deaf adults who rely on seeing spoken language. Visual speech perception (lipreading/speechreading) is not limited to deaf individuals, as almost everyone demonstrates some visual spoken word recognition; but ability varies widely from individual to individual. A mechanistic account at the neural level is needed to help explain lipreading ability, its individual variation, its potential for plasticity, and its role in the spoen language processing system. Based on extensive perceptual research and initial neural evidence, this development project will test a novel hypothesis that visual spoken word representations are stored in the high-level vision cortical pathway. Neuroimaging, computational, and behavioral methods will be used to test this hypothesis in individuals with normal hearing and vision. To test this hypothesis requires overcoming deficiencies in the resolution of conventional functional magnetic resonance imaging (fMRI) data and achieving control over visible spoken stimuli. We will apply advanced neuroimaging techniques (rapid adaptation fMRI and connectivity analyses) and computational modeling of speech dissimilarity to localize in the cerebral cortex activity in response to visible spoken nonsense syllable and word stimuli. Localizers will be used to identify in individuals an area in high-level visual corte that has previously been shown to be selective for visible speech, an area selective for non-speech face motion, areas selective for visual orthographic word forms and for semantic processing, the lateral occipital complex, the fusiform face area, and the human visual motion area. A longterm goal of this project is to determine whether the visual speech pathway is organized the same way as the auditory speech pathway, with multiple levels of representation from speech features to words. The visual speech pathway might be hierarchically shallow, with visual features that are not specific to speech projecting directly to speech representations in high-level vision cortical areas. This project will test specific hypotheses about the selectivity f localized brain areas for visible spoken words and syllables. It will also investigate adaptation, connectivity, and psychophysiological effects in whole brain analyses focused on relationships among brain areas that might contribute to visual perception of speech. The project involves a highly innovative collaboration between speech science researchers and vision scientists using a multidisciplinary approach that can benefit both areas of research. Detailed understanding of how the complex and dynamic visual speech stimulus is processed in the visual pathway will contribute to vision science. Clinical relevance. If evidence is obtained for high-level visual system involvement in speech recognition, innovative clinical methods for training lipreading in those with hearing loss become available from vision science. Individuals with hearing loss frequently depend on visible speech in face-to-face communication, and enhanced ability to use visual information could improve the quality of their lives. Explanations for individual difference in lipreading and methods to improve lipreading have been sought for over a century. In future studies, we can use the proposed methods to isolate causes for lipreading differences. Studies can test the hypothesis that poor lipreading corresponds to poorly tuned neuronal spoken visual word form representations with excessive adaptation for dissimilar and poorly discriminated stimuli, and good lipreading corresponds to minimal neuronal adaptation beyond a certain level of stimulus dissimilarity.