RI: Small: Collaborative Research: Towards Modeling Source Separation from Measured Cortical Responses

RI：小型：协作研究：根据测量的皮质反应对源分离进行建模

• 批准号: 1320260
• 负责人: Nelson Morgan
• 金额: $ 27万
• 依托单位: International Computer Science Institute
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-09-15 至 2016-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1320260&HistoricalAwards=false
• 关键词: RI; Small; Collaborative; Research; Towards

This project will use new technologies for measuring brain activity to understand in detail how human listeners are able to separate competing, overlapping voices, and thereby to help design automatic systems capable of the same feat. Natural environments are full of overlapping sounds, and successful audio processing by both humans and machines relies on a fundamental ability to separate out sound sources of interest. This is commonly referred to as the "cocktail party effect," based on the ability of people to hear what a single person is saying despite the noisy background audio from other speakers. Despite the long history of research in hearing, this exceptional human capability for sound source separation is still poorly understood, and efforts to automatically separate overlapping voices by machine are correspondingly crude: although great advances have been made in robust processing of noisy speech by machine, separation of complex natural sounds (such as overlapping voices) remains a challenge. Advances in sensor technology now enable the modeling of this function in humans, giving an unprecedented, detailed view of sound representation processing in the brain. This project works specifically with measurements of neuroelectric response made directly on the surface of the human cortex (currently with a 256-electrode sensor array) for patients awaiting neurosurgery. Using such measurements made for controlled mixtures of voices, the project will endeavor to both develop models of voice separation in the human cortex by reconstructing an approximation to the acoustic stimulus from the neural population response, and in the process learning the linear mapping between the neural response back to a spectrogram measure of the stimulus. To attempt to significantly improve the ability of machine algorithms to mimic human source separation capability, the project will also focus on a signal processing framework that supports experiments with different combinations of cues and strategies to optimize agreement with the recordings of neural activity. The engineering model is based on the Computational Auditory Scene Analysis (CASA) framework, a family of approaches that have shown competitive results for handling sound mixtures.

该项目将使用新技术来测量大脑活动，以详细了解人类听众如何能够分离竞争，重叠的声音，从而帮助设计具有相同功能的自动系统。自然环境中充满了重叠的声音，人类和机器成功的音频处理依赖于分离感兴趣声源的基本能力。这通常被称为“鸡尾酒会效应”，基于人们能够听到一个人在说什么，尽管来自其他扬声器的嘈杂背景音频。尽管听觉的研究历史悠久，但人类对声源分离的这种特殊能力仍然知之甚少，并且通过机器自动分离重叠语音的努力也相应地粗糙：尽管机器在噪声语音的鲁棒处理方面取得了很大进展，但复杂自然声音（如重叠语音）的分离仍然是一个挑战。传感器技术的进步现在可以对人类的这一功能进行建模，为大脑中的声音表征处理提供了前所未有的详细视图。 该项目专门针对等待神经外科手术的患者，直接在人类皮层表面（目前有256个电极的传感器阵列）上测量神经电反应。使用这种测量控制的声音混合物，该项目将奋进开发语音分离模型在人类皮层通过重建一个近似的声音刺激从神经群体的反应，并在这个过程中学习神经反应之间的线性映射回刺激的频谱图测量。为了尝试显著提高机器算法模仿人类源分离能力的能力，该项目还将专注于一个信号处理框架，该框架支持不同线索和策略组合的实验，以优化与神经活动记录的一致性。该工程模型是基于计算听觉场景分析（CASA）框架，一个家庭的方法，已显示出竞争力的结果处理声音混合。