CAREER: Encoding models for studying attentional and multisensory modulation of the human subcortical auditory system

人类在嘈杂的地方，如繁华的城市街道或忙碌的酒吧里倾听和匡威的能力是非凡的，但也是神秘的。听众是如何从如此响亮的背景噪音中提取出清晰的信号的呢？这个项目的目标是试图了解当我们试图在声学上具有挑战性的条件下进行交流时，听觉通路中的较低大脑区域如何与大脑中的较高听觉皮层区域相互作用。听觉系统的下部位于脑干和中脑，处理声音并将信息发送到更高级别的皮层区域进行分析和理解。大脑皮层也会将信号传回较低的听觉中枢，但这些“自上而下”的信号的性质以及它们如何帮助我们倾听尚不清楚。值得注意的是，在听觉系统中，自上而下的投射比自下而上的投射多。这表明更高的区域通过这些自上而下的投射来塑造传入的听觉信息。理解这些自上而下的信号可以帮助我们理解为什么有些人，即使听力正常，在有很多背景噪音的情况下也很难理解对他们说的话。解开这些自上而下的信号的作用也可以帮助我们理解听觉感知，以及它在多大程度上是由外部声音驱动的，或者是由内部信息（如期望）塑造的，或者是由选择性注意过滤的。除了科学目标之外，该项目还将包括一个综合的研究、教育和推广计划，作为听觉感知和听觉神经科学课程和暑期推广计划的一部分，为高中生和本科生提供动态脑成像的实践经验。为了加深对自上而下投射作用的理解，研究人员将使用脑电图（EEG）测量受试者的大脑活动，同时研究参与者听各种声音，包括滴答声，音乐和演讲。该项目将研究自上而下的注意力如何影响皮层下的听觉处理，方法是为受试者播放两个故事，并要求他们注意其中一个，然后比较大脑对有人注意和无人注意的故事的反应。研究人员还将测试视觉自上而下的信号如何影响皮层下的听觉反应，方法是比较受试者的大脑反应，这些受试者被提供了讲话者的视频与讲话音频匹配或不匹配的讲话录音。使用一组数学工具，称为“编码模型”，可以分析自上而下连接的贡献，以帮助较低的听觉区域处理语音。 研究人员将采用大数据方法，通过使用深度神经网络方法来分析从少数受试者的许多EEG记录中收集的大量大脑数据，开发下一代编码模型。这种方法将为研究人员提供更强大的工具来分析听觉处理和注意力的各个方面。使用这些更复杂的编码模型将阐明较低的听觉中心在感知中发挥的关键作用，揭示自上而下的信号对听觉感知的强烈影响，更普遍地说，最重要的角色-该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准。

The ability of humans to listen and converse in noisy places like bustling city streets or busy bars, is remarkable but also mysterious. How are listeners able to extract a clear signal from such loud background noise? The goal of this project is to try to understand how lower brain areas in the auditory pathway interact with higher auditory cortical areas in the brain when we try to communicate in acoustically challenging conditions. The lower parts of the auditory system, which are located in the brainstem and midbrain, process sounds and send information to higher level cortical areas to be analyzed and understood. The cortex also sends signals back down to the lower auditory centers—but the nature of these “top-down” signals and how they help us listen is not known. Remarkably, there are more top-down projections in the auditory system than “bottom-up” projections. This suggests that higher areas are shaping the incoming auditory information through these top-down projections. Understanding these top-down signals can help us understand why some people, even with normal hearing, struggle to understand what is being said to them when there is a lot of background noise. Untangling the role of these top-down signals can also inform our understanding of auditory perception and how much of it is driven by the external sounds out in the world, or shaped by internal information such expectation, or filtered by selective attention. In addition to the scientific goals, this project will include an integrated research, education, and outreach plan which will give high school students and undergraduates hands-on experiences in dynamic brain imaging as part of classes and summer outreach programs on auditory perception and auditory neuroscience.To deepen understanding of the role of the top-down projections, researchers will measure subjects’ brain activity using electroencephalography (EEG) while study participants listen to a variety of sounds including clicks, music and speech. The project will study how top-down attention affects subcortical auditory processing by playing two stories for subjects and asking them to attend to one of them, and then comparing the brain responses to the attended versus the unattended story. Researchers will also test how visual top-down signals affect subcortical auditory responses by comparing brain responses from subjects presented with recordings of speech where the video of the talker either matches the speech audio or is mismatched. Using a set of mathematical tools that model auditory processing called “encoding models,” it will be possible to analyze the contribution of top-down connections to help lower auditory areas to process the speech sounds. Researchers will take a big data approach and develop the next generation of encoding models by using a deep neural network approach to analyze massive amounts of brain data gathered from many EEG recordings of a small number of subjects over many weeks. This approach will give researchers more powerful tools to analyze aspects of auditory processing and attention. The use of these more sophisticated encoding models will illuminate the critical role lower auditory centers play in perception, reveal the strong impact of top-down signals on auditory perception, and more generally, the role that top-down signals play in all shaping perception from all sensory modalities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.