Crowdsourcing Multi-label Audio Annotation Tasks with Citizen Scientists
Crowdsourcing Multi-label Audio Annotation Tasks with Citizen Scientists
复制标题
与公民科学家一起众包多标签音频注释任务
DOI：
10.1145/3290605.3300522
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
O. Nov
中科院分区：
文献类型：
--
作者：
M. Cartwright;G. Dove;Ana Elisa Méndez Méndez;J. Bello;O. Nov
注释丰富的音频数据是训练和评估机器听力系统的一个重要方面。我们在时间复杂的城市声景背景下完成这项任务，这需要多个标签来识别重叠的声源。通常，这项工作是众包的，之前的研究表明，工作人员可以快速使用单个类的二进制注释来标记音频。然而，当需要具有不同焦点类别的多次传递来用多个标签注释数据时，这种方法可能难以扩展。在公民科学中，任务通常基于图像，注释工作通常在一次传递中同时标记多个类别。本文描述了我们在 Zooniverse 公民科学平台上收集的数据，比较了不同音频注释策略的效率。我们比较了多通道二进制注释、单通道多标签注释和混合方法：分层多通道多标签注释。我们讨论了我们的发现，这些发现支持使用多标签注释，并参考志愿公民科学家的动机。
Annotating rich audio data is an essential aspect of training and evaluating machine listening systems. We approach this task in the context of temporally-complex urban soundscapes, which require multiple labels to identify overlapping sound sources. Typically this work is crowdsourced, and previous studies have shown that workers can quickly label audio with binary annotation for single classes. However, this approach can be difficult to scale when multiple passes with different focus classes are required to annotate data with multiple labels. In citizen science, where tasks are often image-based, annotation efforts typically label multiple classes simultaneously in a single pass. This paper describes our data collection on the Zooniverse citizen science platform, comparing the efficiencies of different audio annotation strategies. We compared multiple-pass binary annotation, single-pass multi-label annotation, and a hybrid approach: hierarchical multi-pass multi-label annotation. We discuss our findings, which support using multi-label annotation, with reference to volunteer citizen scientists' motivations.
“猜猜看！你是第一个看到这个活动的人”：增加对在线制作社区的贡献
DOI：
10.1145/2957276.2957284
发表时间：
2016
期刊：
GROUP '16: Proceedings of the 19th International Conference on Supporting Group Work
影响因子：
--
作者：
Jackson, Corey Brian;Crowston, Kevin;Mugar, Gabriel;Østerlund, Carsten
通讯作者：
Østerlund, Carsten