The role of amplitude envelope in auditory-visual integration

振幅包络在听觉-视觉整合中的作用

• 批准号: 386603-2010
• 负责人: Schutz, Michael
• 金额: $ 1.75万
• 依托单位: McMaster University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2011
• 资助国家: 加拿大
• 起止时间: 2011-01-01 至 2012-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=485423
• 关键词: role; amplitude; envelope; auditory; visual

Ventriloquists routinely exploit the perceptual system's propensity to integrate auditory and visual information, using it to trick audiences into "hearing" puppets speak. I have shown that some musicians (percussionists) use similar principles, allowing their audiences to hear certain musical notes as longer or shorter than their acoustic duration. This novel illusion using naturalistic percussion sounds is at odds with earlier research (often using unnatural, artificial sounds), which has repeatedly concluded that such patterns of visual influence are not possible. Curiously, although artificial sounds are popular in a wide range of auditory research, they rarely occur in the natural world. In contrast, percussive sounds are common in our everyday experience. Therefore this proposed research program will explore the degree to which previous work using artificial sounds generalizes to everyday perceiving by focusing on the following three objectives: (1) Exploring vision's influence on the perception of "hybrid" sounds mixing characteristics of percussive and artificial sounds. (2) Exploring whether percussive sounds are easier to process (one explanation for their propensity to integrate with visual information) by examining reaction times for detecting target intervals amidst a stream of percussive vs. artificial sounds. (3) Testing participants on their ability to intentionally associate sounds with objects, exploring whether performance is better for percussive rather than artificial sounds. Together, these experiments will contribute to our basic understanding of auditory perception by documenting the extent to which previous research using artificial sounds generalizes to the perception of real world auditory events. This work will also be relevant to researchers in other fields. For example, it will be useful to brain scientists investigating the biological basis of multi-sensory perception, an area of neuroscience research that has seen a great deal of growth in recent years. Additionally, our third objective focused on remembering associations between sounds and objects could prove helpful in designing effective auditory human-computer-interfaces, which require users to remember associations between sounds and their intended meaning.

腹语表演者经常利用感知系统整合听觉和视觉信息的倾向，用它来欺骗观众，让他们“听到”木偶说话。我已经展示了一些音乐家（打击乐手）使用类似的原理，让他们的听众听到某些音符比他们的声音持续时间更长或更短。这种使用自然打击声音的新颖错觉与早期的研究（通常使用不自然的、人造的声音）不一致，后者一再得出结论，认为这种视觉影响模式是不可能的。奇怪的是，尽管人造声音在广泛的听觉研究中很受欢迎，但它们很少出现在自然界中。相比之下，打击乐在我们的日常生活中很常见。因此，本研究计划将通过以下三个目标来探索先前使用人工声音的工作在多大程度上推广到日常感知：(1)探索视觉对“混合”声音感知的影响，这些声音是敲击声和人工声音的混合特征。(2)研究敲击声是否更容易处理（一种解释他们倾向于整合视觉信息），通过检查在一系列敲击声和人工声音中检测目标间隔的反应时间。(3)测试参与者有意将声音与物体联系起来的能力，探索敲击声的表现是否比人造声音更好。总之，这些实验将有助于我们对听觉感知的基本理解，通过记录以前使用人工声音的研究在多大程度上概括了对现实世界听觉事件的感知。这项工作对其他领域的研究人员也有一定的借鉴意义。例如，对于研究多感官知觉的生物学基础的脑科学家来说，这将是有用的，这是近年来神经科学研究的一个领域，取得了很大的发展。此外，我们的第三个目标是记住声音和物体之间的联系，这可能有助于设计有效的听觉人机界面，这需要用户记住声音和它们的意图之间的联系。