Visual Form-Based Spoken Word Processing

基于视觉形式的口语文字处理

• 批准号: 8545155
• 负责人: EDWARD T AUER
• 金额: $ 14.54万
• 依托单位: GEORGE WASHINGTON UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-09-13 至 2016-08-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/8545155
• 关键词: Accounting; Adult; Area; Auditory; Behavioral; Brain; Categories; Cerebral cortex; Clinical; Collaborations; Communication; Complex; Computer Simulation; Data; Development; Discrimination; Face; Functional Magnetic Resonance Imaging; Future; Goals; Hearing; Hearing Impaired Persons; Human; Individual; Individual Differences; Inferior frontal gyrus; Language; Lateral; Learning; Left; Lipreading; Measures; Methodology; Methods; Modeling; Motion; Names; Neurons; Orthography; Outcome; Participant; Pathway interactions; Pattern; Process; Psychophysiology; Reporting; Research; Research Personnel; Resolution; Role; Scanning; Science; Scientist; Semantics; Shapes; Signal Transduction; Site; Speech; Speech Perception; Stimulus; Structure of middle temporal gyrus; Structure of superior temporal sulcus; System; Techniques; Testing; Training; Variant; Vision; Visual; Visual Motion; Visual Pathways; Visual Perception; Visual system structure; Word Processing; Work; area MT; base; blood oxygen level dependent; clinically relevant; cognitive neuroscience; density; design; extrastriate visual cortex; fusiform face area; hearing impairment; improved; innovation; interdisciplinary approach; interest; language processing; lexical; multisensory; neuroimaging; novel; novel diagnostics; paired stimuli; physical property; rapid technique; relating to nervous system; research study; response; semantic processing; speech processing; speech recognition; vision science; visual information

DESCRIPTION (provided by applicant): Speech can be learned and perceived on the basis of vision, as demonstrated by prelingually deaf adults who rely on seeing spoken language. Visual speech perception (lipreading/speechreading) is not limited to deaf individuals, as almost everyone demonstrates some visual spoken word recognition; but ability varies widely from individual to individual. A mechanistic account at the neural level is needed to help explain lipreading ability, its individual variation, its potential for plasticity, and its role in the spoen language processing system. Based on extensive perceptual research and initial neural evidence, this development project will test a novel hypothesis that visual spoken word representations are stored in the high-level vision cortical pathway. Neuroimaging, computational, and behavioral methods will be used to test this hypothesis in individuals with normal hearing and vision. To test this hypothesis requires overcoming deficiencies in the resolution of conventional functional magnetic resonance imaging (fMRI) data and achieving control over visible spoken stimuli. We will apply advanced neuroimaging techniques (rapid adaptation fMRI and connectivity analyses) and computational modeling of speech dissimilarity to localize in the cerebral cortex activity in response to visible spoken nonsense syllable and word stimuli. Localizers will be used to identify in individuals an area in high-level visual corte that has previously been shown to be selective for visible speech, an area selective for non-speech face motion, areas selective for visual orthographic word forms and for semantic processing, the lateral occipital complex, the fusiform face area, and the human visual motion area. A longterm goal of this project is to determine whether the visual speech pathway is organized the same way as the auditory speech pathway, with multiple levels of representation from speech features to words. The visual speech pathway might be hierarchically shallow, with visual features that are not specific to speech projecting directly to speech representations in high-level vision cortical areas. This project will test specific hypotheses about the selectivity f localized brain areas for visible spoken words and syllables. It will also investigate adaptation, connectivity, and psychophysiological effects in whole brain analyses focused on relationships among brain areas that might contribute to visual perception of speech. The project involves a highly innovative collaboration between speech science researchers and vision scientists using a multidisciplinary approach that can benefit both areas of research. Detailed understanding of how the complex and dynamic visual speech stimulus is processed in the visual pathway will contribute to vision science. Clinical relevance. If evidence is obtained for high-level visual system involvement in speech recognition, innovative clinical methods for training lipreading in those with hearing loss become available from vision science. Individuals with hearing loss frequently depend on visible speech in face-to-face communication, and enhanced ability to use visual information could improve the quality of their lives. Explanations for individual difference in lipreading and methods to improve lipreading have been sought for over a century. In future studies, we can use the proposed methods to isolate causes for lipreading differences. Studies can test the hypothesis that poor lipreading corresponds to poorly tuned neuronal spoken visual word form representations with excessive adaptation for dissimilar and poorly discriminated stimuli, and good lipreading corresponds to minimal neuronal adaptation beyond a certain level of stimulus dissimilarity.

描述（由申请人提供）：言语可以在视觉的基础上学习和感知，正如依赖于看到口语的语前聋成年人所证明的那样。视觉言语感知（唇读/言语阅读）不仅限于聋人，因为几乎每个人都表现出一些视觉口语单词识别;但能力因人而异。需要一个神经层次的机制来帮助解释唇读能力，它的个体差异，它的可塑性潜力，以及它在spoen语言处理系统中的作用。基于广泛的感知研究和初步的神经证据，该开发项目将测试一个新的假设，即视觉口语表征存储在高级视觉皮层通路中。神经影像学、计算和行为学方法将被用于在听力和视力正常的个体中检验这一假设。为了验证这一假设，需要克服传统的功能性磁共振成像（fMRI）数据的分辨率的缺陷，并实现对可见的口头刺激的控制。我们将应用先进的神经成像技术（快速适应功能磁共振成像和连接分析）和语音差异的计算建模，以定位大脑皮层活动，以响应可见的口语无意义音节和单词刺激。定位器将被用来确定在个人的高层次的视觉corte，以前已被证明是选择性的可见语音，非语音面部运动的区域选择性，区域选择性的视觉正字法的单词形式和语义处理，枕叶外侧复合体，梭状回面部区域，和人类视觉运动区的区域。该项目的长期目标是确定视觉语音通路是否以与听觉语音通路相同的方式组织，具有从语音特征到单词的多层次表示。视觉言语通路可能是层次浅，与视觉特征，不是特定的语音直接投射到语音表征在高级视觉皮层区域。这个项目将测试特定的假设选择性局部大脑区域的可见口语单词和音节。它还将调查适应，连接和心理生理学的影响，在全脑分析集中在大脑区域之间的关系，可能有助于语音的视觉感知。该项目涉及语音科学研究人员和视觉科学家之间的高度创新合作，使用多学科方法，可以使这两个研究领域受益。深入了解复杂动态的视觉言语刺激是如何在视觉通路中被加工的，将有助于视觉科学的发展。临床相关性。如果获得证据表明高级视觉系统参与语音识别，则可以从视觉科学中获得用于训练听力损失患者唇读的创新临床方法。听力损失患者在面对面交流中经常依赖可见语音，增强使用视觉信息的能力可以提高他们的生活质量。世纪以来，人们一直在寻找唇读的个体差异和改善唇读的方法。在未来的研究中，我们可以使用所提出的方法来隔离唇读差异的原因。研究可以测试这一假设，即差的唇读对应于调节不良的神经元口语视觉单词形式表征，过度适应不相似和区分不良的刺激，良好的唇读对应于超过一定水平的刺激相异性的最小神经元适应。