AUDITORY-VISUAL SPEECH PERCEPTION IN INFANTS AND ADULTS

对言语的感知通常被认为是单纯的听觉 现象。然而，这个实验室和其他实验室的研究表明， 对言语的感知不仅仅在于听觉的范畴。 可视语音信息(说话者嘴巴运动的景象) 强烈地影响成年听众的言语知觉。此外， 我们的研究表明，听觉和视觉之间的联系 发音的产品在很小的时候就出现了；婴儿只有18岁 几周大的孩子可以将言语的听觉后果与视觉联系起来 导致它的运动。在上一个资助期内，我们的研究 重点研究了听觉和视觉语音信息的形式 跨模式融合点。我们证明：(A)在……点上 听觉-视觉会聚语音信息没有在语音上 机密；它仍处于前置形式，(B)非常不一致 可以组合听觉和视觉语音信息，例如当 男性的脸和女性的声音结合在一起，(C)专心 机制影响听觉-视觉言语知觉。我们还表明， 为了将听觉和视觉语音信息联系起来，婴儿需要 听觉方面，整个语音信号。这项工作导致了 形成一个新的假设，生成程序性的系列 在这里提出的研究。假设是听觉-视觉言语 信息被映射到特定于语言的存储表示形式 实例化听觉和视觉的语音单位(“原型”) 伴随语音的声音。在下一个资助期，我们将重点放在 三个主题：(A)语音的视听原型-我们将测试 语音原型包括有关发音的信息的概念 是视觉上特定的，(B)对言语原型的跨语言研究 我将调查特定的语言经验对 以两国成年人为研究对象的言语单位表征 (C)婴儿言语原型的发展。 和幼儿-我们将探索言语的形成和本质 个体发育学中的表征。这项拟议的研究对 言语知觉理论，更一般的认知理论 人类发育的分类、模型和临床关注的问题 聋人和盲人群体。

The perception of speech is typically considered a solely auditory phenomenon. However, research in this lab and others shows that the perception of speech does not lie solely in the province of audition. Visible speech information (the sight of a talker's mouth movements) strongly influences the perception of speech in adult listeners. Moreover, our work has shown that connections between the audible and visible products of articulation are present very early in life; infants only 18 weeks old can relate the auditory consequences of speech to the visible movements that cause it. During the last funding period our studies have focused on the form of the auditory and visual speech information at the point of cross-modal integration. We show that: (a) at the point of auditory-visual conflux speech information has not been phonetically classified; it is still in a precategorical form, (b) very discrepant auditory and visual speech information can be combined, such as when a male's face is combined with a female's voice, and (c) that attentional mechanisms influence auditory-visual speech perception. We also show that in order to link auditory and visual speech information, infants need, on the auditory side, the whole speech signal. The work has led to the formation of a new hypothesis that generates the programmatic series of studies proposed here. The hypothesis is that auditory-visual speech information is mapped on to language-specific stored representations of phonetic units ("prototypes") that instantiate both the auditory and visual concomitants of speech sounds. In the next funding period, we focus on three themes: (a) auditory-visual prototypes for speech - we will test the notion that speech prototypes include information about articulation that is visually specified, (b) cross-language studies of speech prototypes - we will investigate the influence of specific language experience on the representation of speech units by studying adults from two countries (America and Japan), and (c) development of speech prototypes in infants and young children - we will explore the formation and nature of speech representations in ontogeny. The proposed research has implications for theories of speech perception, more general theories of cognitive categorization, models of human development, and the clinical concerns of deaf and blind populations.