AUDITORY-VISUAL SPEECH PERCEPTION IN INFANTS AND ADULTS

语音的感知通常被认为是一个单独的听觉 现象 然而，这个实验室和其他实验室的研究表明， 对言语的感知并不仅仅存在于听觉领域。 可视语音信息（说话者的嘴部动作） 强烈影响成年听众的言语感知。 此外，委员会认为， 我们的研究表明，听觉和视觉之间的联系 发音产物在生命的早期就存在;婴儿只有18 几个星期大的孩子就能把言语的听觉后果与视觉后果联系起来， 在上一个资助期内，我们的研究 重点研究了听觉和视觉言语信息的形式， 跨模式整合点。 我们证明：（a）在 视觉-听觉融合的言语信息在语音上没有得到充分的表达， 分类;它仍然是一个precategorical形式，（B）非常不一致 听觉和视觉语音信息可以被组合，例如当 男性的脸与女性的声音相结合，（c）注意力 影响认知-视觉言语知觉的机制。 我们也证明了 为了连接听觉和视觉语言信息，婴儿需要， 听觉方面，整个语音信号。 这项工作导致了 形成一个新的假设，产生了一系列的纲领性的 在此提出的研究。 这个假设是， 信息被映射到特定于语言的存储表示， 语音单位（“原型”），实例化的听觉和视觉 语音的伴随物。 在下一个融资期内，我们将重点关注 三个主题：（a）语音的视觉原型-我们将测试 语音原型包括关于发音的信息， （B）言语原型的跨语言研究--我们 将研究特定语言经验对 对两个国家成年人的言语单位表征的研究 （美国和日本），以及（c）婴儿言语原型的发展 和幼儿-我们将探讨言语的形成和性质 个体发育中的表征 拟议的研究对以下方面有影响： 言语感知理论，更一般的认知理论 分类，人类发展模型，以及临床问题， 聋哑人和盲人。

The perception of speech is typically considered a solely auditory phenomenon. However, research in this lab and others shows that the perception of speech does not lie solely in the province of audition. Visible speech information (the sight of a talker's mouth movements) strongly influences the perception of speech in adult listeners. Moreover, our work has shown that connections between the audible and visible products of articulation are present very early in life; infants only 18 weeks old can relate the auditory consequences of speech to the visible movements that cause it. During the last funding period our studies have focused on the form of the auditory and visual speech information at the point of cross-modal integration. We show that: (a) at the point of auditory-visual conflux speech information has not been phonetically classified; it is still in a precategorical form, (b) very discrepant auditory and visual speech information can be combined, such as when a male's face is combined with a female's voice, and (c) that attentional mechanisms influence auditory-visual speech perception. We also show that in order to link auditory and visual speech information, infants need, on the auditory side, the whole speech signal. The work has led to the formation of a new hypothesis that generates the programmatic series of studies proposed here. The hypothesis is that auditory-visual speech information is mapped on to language-specific stored representations of phonetic units ("prototypes") that instantiate both the auditory and visual concomitants of speech sounds. In the next funding period, we focus on three themes: (a) auditory-visual prototypes for speech - we will test the notion that speech prototypes include information about articulation that is visually specified, (b) cross-language studies of speech prototypes - we will investigate the influence of specific language experience on the representation of speech units by studying adults from two countries (America and Japan), and (c) development of speech prototypes in infants and young children - we will explore the formation and nature of speech representations in ontogeny. The proposed research has implications for theories of speech perception, more general theories of cognitive categorization, models of human development, and the clinical concerns of deaf and blind populations.