Informed Sound Activity Detection in Music and Audio Signals

音乐和音频信号中的明智声音活动检测

• 批准号: 350953655
• 负责人: Dr.-Ing. Jakob Abeßer
• 金额: --
• 依托单位: Fraunhofer-Institut für Digitale Medientechnologie (IDMT)
• 依托单位国家: 德国
• 项目类别: Research Grants
• 财政年份: 2017
• 资助国家: 德国
• 起止时间: 2016-12-31 至 2021-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/350953655?language=en
• 关键词: Informed; Sound; Activity; Detection; Music

In music information retrieval (MIR), the development of computational methods for analyzing, segmenting, and classifying music signals is of fundamental importance. In this project's first phase (initial proposal), we explored fundamental techniques for detecting characteristic sound events present in a given music recording. Here, our focus was on informed approaches that exploit musical knowledge in the form of score information, instrument samples, or musically salient sections. We considered concrete tasks such as locating audio sections with a specific timbre or instrument, identifying monophonic themes in complex polyphonic music recordings, and classifying music genres or playing styles based on melodic contours. We tested our approaches within complex music scenarios, including instrumental Western classical music, jazz, and opera recordings. In the second phase of the project (renewal proposal), our goals will be significantly extended. First, we want to go beyond the music scenario by considering environmental sounds as a second challenging audio domain. As a central methodology, we plan to explore and combine the benefits of model-based and data-driven techniques to learn task-specific sound event representations. Furthermore, we will investigate hierarchical approaches to simultaneously incorporate, exploit, learn, and capture sound events that manifest on different temporal scales and belong to hierarchically ordered categories. An overarching goal of the project's second phase is to develop explainable deep learning models that provide a better understanding of the structural and acoustic properties of sound events.

在音乐信息检索(MIR)中，开发用于分析、分割和分类音乐信号的计算方法是至关重要的。在这个项目的第一阶段(最初的提议)，我们探索了检测给定音乐录音中存在的特征声音事件的基本技术。在这里，我们的重点是以乐谱信息、乐器样本或音乐突出部分的形式利用音乐知识的知情方法。我们考虑了具体的任务，比如用特定的音色或乐器定位音频部分，在复杂的复调音乐录音中识别单音主题，以及根据旋律轮廓对音乐流派或演奏风格进行分类。我们在复杂的音乐场景中测试了我们的方法，包括器乐西方古典音乐、爵士乐和歌剧录音。在项目的第二阶段(续签提案)，我们的目标将显著延长。首先，我们希望超越音乐场景，将环境声音视为第二个具有挑战性的音频领域。作为一种中心方法，我们计划探索并结合基于模型和数据驱动的技术的优点，以学习特定于任务的声音事件表示。此外，我们将研究分层方法，以同时合并、利用、学习和捕获在不同时间尺度上表现的、属于分层有序类别的声音事件。该项目第二阶段的首要目标是开发可解释的深度学习模型，以便更好地了解声音事件的结构和声学特性。