The role of amplitude envelope in auditory-visual integration

口技表演者经常利用知觉系统整合听觉和视觉信息的倾向，利用它来诱骗观众“听到”木偶说话。我已经证明，一些音乐家(打击乐)使用类似的原则，允许他们的观众听到某些音符比他们的声学时长更长或更短。这种使用自然主义敲击声音的新颖错觉与早期的研究(通常使用非自然的人造声音)不一致，后者一再得出结论，这种视觉影响模式是不可能的。奇怪的是，尽管人工声音在广泛的听觉研究中很受欢迎，但它们在自然界中很少出现。相比之下，敲击声在我们的日常体验中很常见。因此，本研究计划将通过以下三个目标来探索以前使用人工声音的工作在多大程度上推广到日常感知：(1)探索视觉对打击声和人工声音混合特征的混合声音知觉的影响。(2)研究敲击声音是否更容易处理(这是敲击声音与视觉信息整合倾向的一种解释)，方法是在一系列敲击声音与人工声音中检测目标间隔的反应时。(3)测试参与者有意将声音与物体联系起来的能力，探索打击乐声音是否比人造声音表现更好。总之，这些实验将有助于我们对听觉感知的基本理解，因为它们记录了以前使用人工声音的研究在多大程度上概括了对真实世界听觉事件的感知。这项工作也将与其他领域的研究人员相关。例如，它将有助于脑科学家研究多感觉感知的生物学基础，这是神经科学研究的一个领域，近年来取得了很大的增长。此外，我们的第三个目标是记住声音和物体之间的联系，这可能有助于设计有效的听觉人机界面，这要求用户记住声音和它们预期意义之间的联系。

Ventriloquists routinely exploit the perceptual system's propensity to integrate auditory and visual information, using it to trick audiences into "hearing" puppets speak. I have shown that some musicians (percussionists) use similar principles, allowing their audiences to hear certain musical notes as longer or shorter than their acoustic duration. This novel illusion using naturalistic percussion sounds is at odds with earlier research (often using unnatural, artificial sounds), which has repeatedly concluded that such patterns of visual influence are not possible. Curiously, although artificial sounds are popular in a wide range of auditory research, they rarely occur in the natural world. In contrast, percussive sounds are common in our everyday experience. Therefore this proposed research program will explore the degree to which previous work using artificial sounds generalizes to everyday perceiving by focusing on the following three objectives: (1) Exploring vision's influence on the perception of "hybrid" sounds mixing characteristics of percussive and artificial sounds. (2) Exploring whether percussive sounds are easier to process (one explanation for their propensity to integrate with visual information) by examining reaction times for detecting target intervals amidst a stream of percussive vs. artificial sounds. (3) Testing participants on their ability to intentionally associate sounds with objects, exploring whether performance is better for percussive rather than artificial sounds. Together, these experiments will contribute to our basic understanding of auditory perception by documenting the extent to which previous research using artificial sounds generalizes to the perception of real world auditory events. This work will also be relevant to researchers in other fields. For example, it will be useful to brain scientists investigating the biological basis of multi-sensory perception, an area of neuroscience research that has seen a great deal of growth in recent years. Additionally, our third objective focused on remembering associations between sounds and objects could prove helpful in designing effective auditory human-computer-interfaces, which require users to remember associations between sounds and their intended meaning.