Excellence in Research: Incorporating Attention into Computational Auditory Scene Analysis Using Spectral Clustering with Focal Templates
卓越研究:使用带有焦点模板的谱聚类将注意力纳入计算听觉场景分析
基本信息
- 批准号:2100874
- 负责人:
- 金额:$ 49.9万
- 依托单位:
- 依托单位国家:美国
- 项目类别:Standard Grant
- 财政年份:2021
- 资助国家:美国
- 起止时间:2021-06-01 至 2024-05-31
- 项目状态:已结题
- 来源:
- 关键词:
项目摘要
Humans display an uncanny ability to focus on a sound of interest, even in the presence of interfering sounds or noise. For instance, a child’s voice may be immediately recognized and reacted-to in the context of parental protection, a bird-watcher may recognize and follow the call of a house finch, or business associates may discuss the latest events over lunch in a crowded diner. In each of these examples, a person directs his or her attention to the sound source of interest. Once attention is directed, a human can continue to focus on that sound, even to the degree that he or she may not perceive (or is able to passively ignore) other sounds. Computational listening systems, at present, are unable to replicate that ability. If such ability was possible, computational listening systems could contribute to an array of application domains. Current hearing aid technology is woefully inadequate in group settings, where noise often overwhelms the listener and the listener is unable to compensate. Those who suffer from hearing loss feel socially isolated, leading to a lower quality of life. Performance of hearing aids, and the lives of those who rely on them, would improve drastically if these devices could automatically adapt to isolate and focus on salient sound sources. Effective acoustic monitoring could be deployed to safety-related applications, such as automatically detecting slurred speech from someone suffering a stroke, or to alert the deaf to important loudspeaker announcements in a public place. Autonomous devices (robots, drones, etc.) could employ enhanced listening techniques to guide their movements or actions, potentially better serving or protecting the public. The ability to focus on a particular instrument within a musical ensemble could lead to improved automatic music transcription systems and enhanced tools for performance analysis and training. The applications of computational listening systems exhibiting perceptually-based attention are boundless.This research will investigate the use of focal templates to incorporate attention into computational auditory scene analysis (CASA). CASA attempts to reproduce (via computational algorithms and systems) the ability of humans to perceive an acoustic scene given only auditory input, and the underlying methods of CASA are modeled upon psychological principles of perception. A focal template may be considered a type of dynamic time-frequency filter that passes only those auditory elements conforming to a pattern of interest. Sound events of interest (if present in the audio) will be detected using spectral clustering, which has emerged as a useful technique for grouping "like" elements within a set; here, the goal is to group auditory elements that pass through the focal template. This work will: 1) implement a model of attention into CASA through use of spectral clustering with focal templates, 2) measure the impact of incorporating attention on the performance of CASA systems in isolating particular sounds of interest, and 3) determine effective methods to develop focal templates while considering how to scale to the "general" listening case. This project includes development of an academic minor in "Sound and Music Computing", a unique educational opportunity to prepare students to contribute to the research while integrating across STEM and non-STEM disciplines. An outcome of this work is to build partnerships and strengthen the Computational Research on Music & Audio Team of Interdisciplinary Collaborators (CRoMA-TIC). Through these activities, the project will: 1) advance the state of the art in CASA, 2) foster collaborative relationships within this field, 3) develop a well-recognized area of expertise and research capacity at Lincoln University, and 4) develop a pipeline for undergraduate students leading to related careers or graduate study.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
人类表现出一种不可思议的能力,即使在干扰声音或噪音的存在下,也能专注于感兴趣的声音。 例如,在父母保护的背景下,孩子的声音可以立即被识别并做出反应,观鸟者可以识别并跟随家雀的叫声,或者商业伙伴可以在拥挤的晚餐中讨论最新的事件。 在这些例子中的每一个中,一个人将他或她的注意力引导到感兴趣的声源。 一旦注意力被引导,人类可以继续专注于那个声音,甚至到他或她可能无法感知(或能够被动忽略)其他声音的程度。 目前,计算听力系统无法复制这种能力。 如果这种能力是可能的,计算听系统可以有助于一系列的应用领域。 目前的助听器技术在群体环境中是远远不够的,在群体环境中,噪音经常困扰听者,听者无法补偿。 听力损失患者会感到社会孤立,导致生活质量下降。 如果助听器能够自动适应隔离并专注于突出的声源,助听器的性能以及依赖它们的人的生活将大大改善。 有效的声学监测可以部署到与安全相关的应用中,例如自动检测中风患者的口齿不清,或者提醒聋人注意公共场所的重要扬声器广播。 自主设备(机器人、无人机等)可以使用增强的监听技术来指导他们的动作或行动,从而更好地服务或保护公众。 在音乐合奏中专注于特定乐器的能力可能会导致改进的自动音乐转录系统和增强的性能分析和培训工具。 基于感知的注意的计算听觉系统的应用是无限的,本研究将探讨焦点模板的使用,将注意纳入计算听觉场景分析(CASA)。 CASA试图(通过计算算法和系统)再现人类感知听觉场景的能力,CASA的基本方法是基于感知的心理学原理。 焦点模板可以被认为是一种动态时频滤波器,其仅通过符合感兴趣模式的那些听觉元素。 感兴趣的声音事件(如果存在于音频中)将使用谱聚类来检测,谱聚类已经成为一种有用的技术,用于对集合内的“相似”元素进行分组;在这里,目标是对通过焦点模板的听觉元素进行分组。 这项工作将:1)通过使用具有焦点模板的谱聚类将注意力模型实现到CASA中,2)测量在分离感兴趣的特定声音时将注意力并入对CASA系统的性能的影响,以及3)确定开发焦点模板的有效方法,同时考虑如何缩放到“一般”收听情况。 该项目包括“声音和音乐计算”的学术未成年人的发展,这是一个独特的教育机会,可以帮助学生为研究做出贡献,同时整合STEM和非STEM学科。 这项工作的成果是建立伙伴关系,加强跨学科合作者音乐音频团队的计算研究(CRoMA-TIC)。 通过这些活动,该项目将:1)推进CASA的最新技术,2)促进该领域内的合作关系,3)在林肯大学发展公认的专业知识和研究能力领域,和4)为本科生发展相关职业或研究生学习的管道。该奖项反映了NSF的法定使命,并被认为是值得支持的,使用基金会的知识价值和更广泛的影响审查标准进行评估。
项目成果
期刊论文数量(1)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Visually Exploring Multi-Purpose Audio Data
可视化探索多用途音频数据
- DOI:10.1109/mmsp53017.2021.9733552
- 发表时间:2021
- 期刊:
- 影响因子:0
- 作者:Heise, David;Bear, Helen L.
- 通讯作者:Bear, Helen L.
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
David Heise其他文献
Acoustically Tracking the Comings and Goings of Bumblebees
用声音追踪大黄蜂的来去
- DOI:
10.1109/sas.2019.8705973 - 发表时间:
2019 - 期刊:
- 影响因子:0
- 作者:
David Heise;Zach Miller;Ellie Harrison;A. Gradišek;J. Grad;C. Galen - 通讯作者:
C. Galen
Pollination on the Dark Side: Acoustic Monitoring Reveals Impacts of a Total Solar Eclipse on Flight Behavior and Activity Schedule of Foraging Bees
黑暗面授粉:声学监测揭示日全食对觅食蜜蜂的飞行行为和活动时间表的影响
- DOI:
10.1093/aesa/say035 - 发表时间:
2018 - 期刊:
- 影响因子:2.3
- 作者:
C. Galen;Zach Miller;A. Lynn;Michael Axe;Samuel Holden;Levi Storks;Eddie Ramirez;Emilia Asante;David Heise;S. Kephart;Jim Kephart - 通讯作者:
Jim Kephart
Evaluating the Potential and Realized Impact of Data Augmentations
评估数据增强的潜在和已实现的影响
- DOI:
- 发表时间:
2023 - 期刊:
- 影响因子:0
- 作者:
David Heise;Helen L. Bear - 通讯作者:
Helen L. Bear
Volume Information
成交量信息
- DOI:
10.1086/ajs.95.6.2780327 - 发表时间:
1990 - 期刊:
- 影响因子:4.4
- 作者:
P. Abell;A. Cicourel;N. Fraser;Mark Abrahamson;D. Clawson;Howard;E.;Freeman;Ronald;Ľ.;Akers;Élisabeth;Clemens;J. Freeman;D. Alwin;W. Clement;N. Friedkin;E. Amenta;R. Clignet;Gerald Friedman;T.;Aminzade;Jere M. Cohen;William;Á.;Gamson;Ronald Angel;S. Cohn;R. Gartner;M. Archer;M. Colvin;Michael;Geyer;R. Axelrod;K. Cook;A. Giddens;W. Baker;Corsaro;T. Gieryn;Karen Barkey;J. Corzine;R. A.;Gillis;James;Ñ.;Baron;Lin Corzine;Todd Gitlin;Richard;Barrett;D. Crane;J. Glass;Frank;D.;Bean;Davis;Norval;Glenn;Gary;Š.;Becker;N. Denton;Thomas;B.;Gold;A. Bickford;Frédéric;C.;Deyo;Francés;P. Birkeland;T. DiPrete;Goldscheider.;Grant;U.;Blank;F. Dobbin;Heidi;Gottfried;Anthony;J.;Blasi;Katherine;Donato;Burke;Grandjean;J. Blau;Starkey Duncan;Mark S. Granovetter;Fred;Block;Lauren;Edelman;Andrew;M.;Greeley;R. Blumberg;D. Elbein;S. Greenhalgh;Lawrence D. Bobo;R. Emigh;L. Griffin;Kenneth;Bollen;Jeanne Enders;David;Grusky;Christine;Bose;W. Espeland;Hachen;T. Boswell;Amitai Etzioni;Y. Y. Haddad;David Brain;Fararo;J. Hagan;W. Bridges;G. Farkas;W. Hagstrom;Steven;G.;Brint;Reynolds;Farley;Valérie;Haines;Charles;Brody;Robert;R.;Faulkner;Archibald;0.;Haller;Kevin Brown;Joe;Feagin;Terence;Halliday;Kevin;Brown;M. Felson;Phillip;Hammond;William I. Brustein;Felson;John;Hartman;L. Bumpass;M. Fennell;David Hawkins;James Burk;Théodore;Douglas;P. Burstein;Ferdinand;Heckathorn;Frédérick;Buttell;M. Roberto;Peter;E. M.;F. Cancian;Fernández;Hedstrom;J. Saltzman;Patrícia;Karen;Hegtvedt;Chafetz;FERNANDEZ;David Heise;A. Cherlin;Fine;B. Heyns;D. Chirot;R. Finke;Alexander;Hicks;Kevin J. Christiano;R. Fishman;Stephen;Hilgartner;Ajs - 通讯作者:
Ajs
Asserting the inherent benefits of hands-on laboratory projects vs. computer simulations
断言动手实验室项目与计算机模拟的固有优势
- DOI:
- 发表时间:
2006 - 期刊:
- 影响因子:0
- 作者:
David Heise - 通讯作者:
David Heise
David Heise的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('David Heise', 18)}}的其他基金
Catalyst Project: Computational Research On Music and Audio
催化剂项目:音乐和音频的计算研究
- 批准号:
1410586 - 财政年份:2015
- 资助金额:
$ 49.9万 - 项目类别:
Standard Grant
相似国自然基金
Research on Quantum Field Theory without a Lagrangian Description
- 批准号:24ZR1403900
- 批准年份:2024
- 资助金额:0.0 万元
- 项目类别:省市级项目
Cell Research
- 批准号:31224802
- 批准年份:2012
- 资助金额:24.0 万元
- 项目类别:专项基金项目
Cell Research
- 批准号:31024804
- 批准年份:2010
- 资助金额:24.0 万元
- 项目类别:专项基金项目
Cell Research (细胞研究)
- 批准号:30824808
- 批准年份:2008
- 资助金额:24.0 万元
- 项目类别:专项基金项目
Research on the Rapid Growth Mechanism of KDP Crystal
- 批准号:10774081
- 批准年份:2007
- 资助金额:45.0 万元
- 项目类别:面上项目
相似海外基金
Developing and testing a novel self-guided digital therapeutic solution for preventing stammering in children: incorporating latest research on early detection and progress evaluation using real-world data
开发和测试一种新颖的自我引导数字治疗解决方案,用于预防儿童口吃:结合使用真实世界数据进行早期检测和进展评估的最新研究
- 批准号:
10072187 - 财政年份:2023
- 资助金额:
$ 49.9万 - 项目类别:
Collaborative R&D
Incorporating Diversity in Alzheimer's Disease Research: Developing Representative and Generalizable models
将多样性纳入阿尔茨海默病研究:开发代表性和可推广的模型
- 批准号:
495662 - 财政年份:2023
- 资助金额:
$ 49.9万 - 项目类别:
Operating Grants
Collaborative Research: Incorporating secondary foundation species in coastal restoration efforts to increase ecosystem regrowth, biodiversity recovery and climate resistance
合作研究:将次要基础物种纳入沿海恢复工作,以促进生态系统再生、生物多样性恢复和气候抵抗力
- 批准号:
2301960 - 财政年份:2023
- 资助金额:
$ 49.9万 - 项目类别:
Standard Grant
Incorporating Communities of Practice and Research Experiences into the Preparation of High School STEM Teachers
将实践和研究经验社区纳入高中 STEM 教师的培养
- 批准号:
2243491 - 财政年份:2023
- 资助金额:
$ 49.9万 - 项目类别:
Continuing Grant
Collaborative Research: Incorporating secondary foundation species in coastal restoration efforts to increase ecosystem regrowth, biodiversity recovery and climate resistance
合作研究:将次要基础物种纳入沿海恢复工作,以促进生态系统再生、生物多样性恢复和气候抵抗力
- 批准号:
2301959 - 财政年份:2023
- 资助金额:
$ 49.9万 - 项目类别:
Standard Grant
Incorporating Quantitative Analysis and Digital Database Use in Structure and Tectonics Research and Teaching: Proposal for a Summer School
将定量分析和数字数据库的使用纳入结构和构造研究与教学:关于暑期学校的建议
- 批准号:
2222610 - 财政年份:2023
- 资助金额:
$ 49.9万 - 项目类别:
Standard Grant
Collaborative Research: Advancing Equity Goals in Secondary Classrooms by Incorporating Mathematical Justification in Teaching
合作研究:通过在教学中纳入数学论证来推进中学课堂的公平目标
- 批准号:
2201858 - 财政年份:2022
- 资助金额:
$ 49.9万 - 项目类别:
Standard Grant
Collaborative Research: Incorporating SPECFEM3D numerical seismograms in the Global CMT Project
合作研究:将 SPECFEM3D 数值地震图纳入全球 CMT 项目
- 批准号:
2218859 - 财政年份:2022
- 资助金额:
$ 49.9万 - 项目类别:
Standard Grant
Incorporating a course-based research experience in high school and community college courses in Southeast Nebraska
在内布拉斯加州东南部的高中和社区大学课程中融入基于课程的研究经验
- 批准号:
2202070 - 财政年份:2022
- 资助金额:
$ 49.9万 - 项目类别:
Standard Grant
New Mexico Integrative Science Program Incorporating Research in Environmental Sciences (NM-INSPIRES)
新墨西哥州综合科学计划纳入环境科学研究(NM-INSPIRES)
- 批准号:
10689665 - 财政年份:2022
- 资助金额:
$ 49.9万 - 项目类别: