The role of amplitude envelope in auditory-visual integration

腹语表演者经常利用感知系统整合听觉和视觉信息的倾向，用它来欺骗观众“听”木偶说话。我已经证明，一些音乐家（音乐家）使用类似的原则，让他们的观众听到某些音符比他们的声学持续时间更长或更短。这种使用自然主义敲击声的新错觉与早期的研究（通常使用不自然的人造声音）不一致，后者一再得出结论，这种视觉影响模式是不可能的。奇怪的是，尽管人工声音在广泛的听觉研究中很受欢迎，但它们很少出现在自然世界中。 相比之下，在我们的日常经验中，重复的声音很常见。 因此，本研究计划将通过以下三个目标来探索以前使用人工声音的工作在多大程度上推广到日常感知：（1）探索视觉对“混合”声音感知的影响，混合声音和人工声音的混合特性。 (2)通过检查在一系列重复声音与人工声音中检测目标间隔的反应时间，探索重复声音是否更容易处理（它们倾向于与视觉信息整合的一种解释）。(3)测试参与者有意识地将声音与物体联系起来的能力，探索模仿声音的表现是否比人工声音更好。总之，这些实验将有助于我们对听觉感知的基本理解，通过记录以前使用人工声音的研究在多大程度上推广到对真实的世界听觉事件的感知。这项工作也将与其他领域的研究人员有关。 例如，它将有助于脑科学家研究多感官知觉的生物学基础，这是近年来神经科学研究的一个领域。 此外，我们的第三个目标专注于记住声音和物体之间的关联，这可能有助于设计有效的听觉人机界面，这需要用户记住声音和它们的预期含义之间的关联。

Ventriloquists routinely exploit the perceptual system's propensity to integrate auditory and visual information, using it to trick audiences into "hearing" puppets speak. I have shown that some musicians (percussionists) use similar principles, allowing their audiences to hear certain musical notes as longer or shorter than their acoustic duration. This novel illusion using naturalistic percussion sounds is at odds with earlier research (often using unnatural, artificial sounds), which has repeatedly concluded that such patterns of visual influence are not possible. Curiously, although artificial sounds are popular in a wide range of auditory research, they rarely occur in the natural world. In contrast, percussive sounds are common in our everyday experience. Therefore this proposed research program will explore the degree to which previous work using artificial sounds generalizes to everyday perceiving by focusing on the following three objectives: (1) Exploring vision's influence on the perception of "hybrid" sounds mixing characteristics of percussive and artificial sounds. (2) Exploring whether percussive sounds are easier to process (one explanation for their propensity to integrate with visual information) by examining reaction times for detecting target intervals amidst a stream of percussive vs. artificial sounds. (3) Testing participants on their ability to intentionally associate sounds with objects, exploring whether performance is better for percussive rather than artificial sounds. Together, these experiments will contribute to our basic understanding of auditory perception by documenting the extent to which previous research using artificial sounds generalizes to the perception of real world auditory events. This work will also be relevant to researchers in other fields. For example, it will be useful to brain scientists investigating the biological basis of multi-sensory perception, an area of neuroscience research that has seen a great deal of growth in recent years. Additionally, our third objective focused on remembering associations between sounds and objects could prove helpful in designing effective auditory human-computer-interfaces, which require users to remember associations between sounds and their intended meaning.