DialCrowd 2.0: A Quality-Focused Dialog System Crowdsourcing Toolkit
DialCrowd 2.0: A Quality-Focused Dialog System Crowdsourcing Toolkit
复制标题
DOI：
10.48550/arxiv.2207.12551
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Jessica Huynh;Ting-Rui Chiang;Jeffrey P. Bigham;M. Eskénazi
中科院分区：
文献类型：
--
作者：
Jessica Huynh;Ting-Rui Chiang;Jeffrey P. Bigham;M. Eskénazi
Dialog系统开发人员需要高质量的数据来训练、微调和评估他们的系统。他们经常使用众包，因为它提供了来自许多工人的大量数据。然而，数据的质量可能不够好。这可能是由于请求者提出任务的方式以及他们与工作者的交互方式。本文介绍了DialCrowd 2.0，以帮助请求者获得更高质量的数据，例如，更清楚地提出任务，并促进与工人的有效沟通。DialCrowd 2.0指导开发人员创建改进的人工智能任务（HIT），并直接适用于开发人员和研究人员当前使用的工作流程。
Dialog system developers need high-quality data to train, fine-tune and assess their systems. They often use crowdsourcing for this since it provides large quantities of data from many workers. However, the data may not be of sufficiently good quality. This can be due to the way that the requester presents a task and how they interact with the workers. This paper introduces DialCrowd 2.0 to help requesters obtain higher quality data by, for example, presenting tasks more clearly and facilitating effective communication with workers. DialCrowd 2.0 guides developers in creating improved Human Intelligence Tasks (HITs) and is directly applicable to the workflows used currently by developers and researchers.