Onsets, Activity, and Events: A Multi-task Approach for Polyphonic Sound Event Modelling
Onsets, Activity, and Events: A Multi-task Approach for Polyphonic Sound Event Modelling
复制标题
DOI：
10.33682/sm6r-8p49
复制
发表时间：
2019-10
期刊：
影响因子：
--
通讯作者：
Arjun Pankajakshan;Helen L. Bear;Emmanouil Benetos;Events
中科院分区：
文献类型：
--
作者：
Arjun Pankajakshan;Helen L. Bear;Emmanouil Benetos;Events
目前最先进的复调声事件检测（SED）系统的功能是框架级多标签分类模型。在每一帧的动态复调水平的背景下，声音事件相互干扰，这降低了分类器学习单个声音事件的准确频率分布的能力。帧级局部分类器也不能明确地模拟声音事件的长期时间结构。因此，事件检测的性能低于分段检测。我们将“时间精确的复调声音事件检测”定义为检测具有正确起始的声音事件实例的子任务。在这里，我们研究了声音活动检测（SAD）和开始检测作为辅助任务，使用多任务学习来提高复音SED的时间精度的有效性。SAD有助于将事件活动帧与噪声帧和沉默帧区分开来，并有助于避免每一帧的漏检。发病预测确保了每个事件的开始，这反过来又用于SAD和SED的条件预测。我们在URBAN-SED数据集上的实验表明，通过将SED与发作检测和SAD相结合，基于事件的F -得分相对提高了三倍以上。
State of the art polyphonic sound event detection (SED) systems function as frame-level multi-label classification models. In the context of dynamic polyphony levels at each frame, sound events interfere with each other which degrade a classifier’s ability to learn the exact frequency profile of individual sound events. Frame-level localized classifiers also fail to explicitly model the long-term temporal structure of sound events. Consequently, the event-wise detection performance is less than the segment-wise detection. We define ‘temporally precise polyphonic sound event detection’ as the subtask of detecting sound event instances with the correct onset. Here, we investigate the effectiveness of sound activity detection (SAD) and onset detection as auxiliary tasks to improve temporal precision in polyphonic SED using multi-task learning. SAD helps to differentiate event activity frames from noisy and silence frames and helps to avoid missed detections at each frame. Onset predictions ensure the start of each event which in turn are used to condition predictions of both SAD and SED. Our experiments on the URBAN-SED dataset show that by conditioning SED with onset detection and SAD, there is over a three-fold relative improvement in event-based F -score.