Progression in a Language Annotation Game with a Purpose
Progression in a Language Annotation Game with a Purpose
复制标题
有目的的语言注释游戏的进展
DOI：
10.1609/hcomp.v7i1.5276
复制
发表时间：
2019
影响因子：
12
通讯作者：
Massimo Poesio
中科院分区：
文献类型：
--
作者：
Chris Madge;Juntao Yu;Jon Chamberlain;Udo Kruschwitz;Silviu Paun;Massimo Poesio
在传统的游戏设计中，融入渐进难度被认为是至关重要的。但是，尽管人们普遍认为，进展在有目的游戏（GWAP）中具有明显的好处——例如，训练非专家注释者做出更复杂的判断——事实上，进展并不是 GWAP 的一个显着特征；关于其影响的证据就更少了。在这项工作中，我们提出了一种 GWAP 进展方法，该方法可推广到不同的注释任务，并且对黄金注释数据的依赖性最小（如果有的话）。使用这种方法，我们观察到与随机向注释者显示项目相比，准确率在统计上显着提高。
Within traditional games design, incorporating progressive difficulty is considered of fundamental importance. But despite the widespread intuition that progression could have clear benefits in Games-With-A-Purpose (GWAPs)–e.g., for training non-expert annotators to produce more complex judgements– progression is not in fact a prominent feature of GWAPs; and there is even less evidence on its effects. In this work we present an approach to progression in GWAPs that generalizes to different annotation tasks with minimal, if any, dependency on gold annotated data. Using this method we observe a statistically significant increase in accuracy over randomly showing items to annotators.