Biocomplexity Incubation Activity: Coordination of Perception and Spoken Language Processes

生物复杂性孵化活动：感知和口语过程的协调

• 批准号: 0083865
• 负责人: Edward Auer
• 金额: $ 4.62万
• 依托单位: House Ear Institute
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2000
• 资助国家: 美国
• 起止时间: 2000-09-15 至 2003-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0083865&HistoricalAwards=false
• 关键词: Biocomplexity; Incubation; Activity; Coordination; Perception

Human speech processing engages multiple neural resources, exhibits individual neural reorganization and variation in response to system perturbations/disorders, evolves along and is also constrained by temporal processes, and responds to statistical properties in the language environment. Human speech perception comprises not only auditory perceptual processes but also visual ones. For example, seeing a talker enhances the ability to accurately perceive speech in noise. Auditory and visual processing of the speech signal is a biocomplex phenomenon. During audiovisual speech perception, information is integrated across perceptual systems to give rise to unitary speech percepts. This integration implies the coordination of perceptual, psycholinguistic, and possibly cognitive systems such as attention. Perception emerges from processing at the levels of cortical and sub-cortical brain structures, but perception cannot be reduced simply to its underlying neural mechanisms, it must also be accounted for in behavioral terms. Speech perception emerges across temporal scales, including the scale of development and the scales of communication. Speech perception also is responsive to environmental demands, as is evident in the development of perceptual skills by expert deaf lipreaders. This Biocomplexity Incubation Activity award will support a project that seeks to create a development context for researchers who might solve the problem of how the auditory and visual perceptual systems are integrated with the spoken language system, but who have not yet had the opportunity to learn enough about each other's fields and perspectives to propose a research program. Researchers with expertise in visual psychophysics and perception, multimodal integration, auditory and visual speech perception, neuroanatomy and neurophysiology, and computational modeling will meet in a regularly scheduled research seminar covering visual and auditory perception in relation to spoken language processing. The seminar will include demonstrations of relevant phenomena and experimental methods/apparatus. Demonstrations for the seminar are intended to teach and also generate discussion. They will include examples such as the McGurk effect, sinewave speech, point light speech, random-dot motion effects, structure from motion and others. The research seminar will be assisted by an undergraduate student who will help to create the demonstrations. These demonstrations will be made available on the House Ear Institute (HEI) Web site. The seminar will develop hypotheses and methods to form the basis for future collaborative research. The foci will be auditory perception, visual perception, audiovisual integration, and visual speech perception enhancement. A workshop will conclude the research seminar. Its purposes are to present the generated hypotheses and methods and to obtain feedback. Presentations will outline the biocomplexity issues addressed in the seminar, the key lines of research identified to be of critical importance, and the experimental and computational approaches following those lines. Expert panelists from other Southern California institutions will participate in the workshop. A final report on the project and the workshop will be prepared for the HEI Web site.The use of spoken language is a fundamental distinguishing characteristic of humans, but knowledge is incomplete regarding how humans accomplish it. The particular focus of this project is to understand how the perceptual systems of vision and audition process linguistically relevant information. A complete understanding of perception and language will need to comprise links between levels of explanation, from the neural to the behavioral. Societal use of this fundamental knowledge will be in such areas as that of enhancing learning and teaching, ameliorating the effects of diseases such as stroke and impairments such as hearing loss, and developing artificial systems for processing language (e.g., automatic speech recognition and machine language translation).

人类语音处理涉及多个神经资源，响应于系统扰动/紊乱而表现出个体神经重组和变化，沿着演化并且还受到时间过程的约束，并且响应于语言环境中的统计特性。 人类的言语感知不仅包括听觉感知过程，还包括视觉感知过程。 例如，看到说话者增强了在噪声中准确感知语音的能力。 语音信号的听觉和视觉处理是一种复杂的生物现象。 在视听语音感知过程中，信息在感知系统中被整合，产生单一的语音感知。 这种整合意味着知觉、心理语言和可能的认知系统（如注意力）的协调。 知觉产生于大脑皮层和皮层下结构的处理过程，但知觉不能简单地归结为其潜在的神经机制，它还必须从行为的角度来解释。 言语知觉的出现跨越了时间尺度，包括发展的尺度和交流的尺度。 言语感知也会对环境需求做出反应，这一点在聋人唇读专家的感知技能发展中显而易见。 这个生物复杂性孵化活动奖将支持一个项目，该项目旨在为研究人员创造一个发展环境，这些研究人员可能解决听觉和视觉感知系统如何与口语系统相结合的问题，但他们还没有机会充分了解对方的领域和观点来提出研究计划。 具有视觉心理物理学和感知，多模态整合，听觉和视觉语音感知，神经解剖学和神经生理学以及计算建模专业知识的研究人员将在定期举行的研究研讨会上会面，该研讨会涵盖与口语处理相关的视觉和听觉感知。 研讨会将包括相关现象和实验方法/仪器的演示。 研讨会的演示旨在进行教学，并引发讨论。 它们将包括诸如McGurk效应、正弦波语音、点光语音、随机点运动效应、运动结构等示例。 研究研讨会将由一名本科生协助，他将帮助创建演示。 这些演示将在House Ear Institute（HEI）网站上提供。 研讨会将发展假设和方法，为未来的合作研究奠定基础。 重点将是听觉感知，视觉感知，视听整合和视觉语言感知增强。 研讨会结束时将举办一个讲习班。 其目的是提出所产生的假设和方法，并获得反馈。 演讲将概述研讨会上讨论的生物复杂性问题，确定为至关重要的研究的关键线，以及遵循这些线的实验和计算方法。 来自南加州其他机构的专家小组成员将参加研讨会。 关于这个项目和研讨会的最终报告将在HEI网站上发表。使用口语是人类的一个基本特征，但关于人类如何完成口语的知识还不完整。这个项目的特别重点是了解视觉和听觉的感知系统如何处理语言相关信息。 对感知和语言的完整理解需要包括从神经到行为的解释层次之间的联系。 这些基本知识的社会应用将在以下领域：加强学习和教学，改善中风等疾病和听力损失等障碍的影响，以及开发处理语言的人工系统（例如，自动语音识别和机器语言翻译）。