Muse-ing on the Impact of Utterance Ordering on Crowdsourced Emotion Annotations
Muse-ing on the Impact of Utterance Ordering on Crowdsourced Emotion Annotations
复制标题
思考话语排序对众包情感注释的影响
DOI：
10.1109/icassp.2019.8682793
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
E. Provost
中科院分区：
文献类型：
--
作者：
Mimansa Jaiswal;Zakaria Aldeneh;Cristian;Y. Luo;Mihai Burzo;Rada Mihalcea;E. Provost
情感识别算法依赖于用高质量标签注释的数据。然而，情感表达和感知本质上是主观的。通常没有一个注释可以明确地声明为“正确的”。“因此，注释是由它们被收集的方式着色的。在本文中，我们进行众包实验，调查这两个注释本身和这些算法的性能的影响。我们关注一个关键问题：语境的影响。我们提出了一个新的情感数据集，多模态应激情感（MuSE），并注释数据集使用两个条件：随机化，其中注释者以随机顺序呈现剪辑，和上下文化，其中注释者以顺序呈现剪辑。我们发现，上下文标记方案的结果注释更类似于扬声器自己的自我报告的标签，从随机方案生成的标签是最容易预测的自动化系统。
Emotion recognition algorithms rely on data annotated with high quality labels. However, emotion expression and perception are inherently subjective. There is generally not a single annotation that can be unambiguously declared "correct." As a result, annotations are colored by the manner in which they were collected. In this paper, we conduct crowdsourcing experiments to investigate this impact on both the annotations themselves and on the performance of these algorithms. We focus on one critical question: the effect of context. We present a new emotion dataset, Multimodal Stressed Emotion (MuSE), and annotate the dataset using two conditions: randomized, in which annotators are presented with clips in random order, and contextualized, in which annotators are presented with clips in order. We find that contextual labeling schemes result in annotations that are more similar to a speaker’s own self-reported labels and that labels generated from randomized schemes are most easily predictable by automated systems.
DOI：
10.1016/0005-7916(94)90063-9
发表时间：
1994-03-01
影响因子：
1.8
作者：
BRADLEY, MM;LANG, PJ
通讯作者：
LANG, PJ