Coordinating human and machine intelligence to classify microblog communications in crises
Coordinating human and machine intelligence to classify microblog communications in crises
复制标题
协调人类和机器智能对危机中的微博通信进行分类
DOI：
--
复制
发表时间：
2014
期刊：
影响因子：
--
通讯作者：
Jakob Rogstadius
中科院分区：
文献类型：
--
作者：
Muhammad Imran;Carlos Castillo;Jesse Lucas;P. Meier;Jakob Rogstadius
一种新兴的数据流处理范例涉及人类和机器计算协同工作，允许人类智能处理大规模数据。我们将这种方法应用于微博流中危机相关信息的分类。我们首先描述AIDR（灾难响应人工智能）平台，该平台随时间收集人类注释，以创建和维护社交媒体消息的自动监督分类器。接下来，我们将研究其设计中的两个重大挑战：(1)确定哪些元素必须由人类标记，以及(2)确定何时要求完成此类注释。第一个挑战是由众包工人选择要贴标签的物品，以最大限度地提高他们的工作效率。第二个挑战是安排工作，以便随着时间的推移可靠地保持高分类精度。我们通过在现实世界的数据集上进行广泛的实验来提供和验证这些挑战的答案。
An emerging paradigm for the processing of data streams involves human and machine computation working together, allowing human intelligence to process large-scale data. We apply this approach to the classification of crisis-related messages in microblog streams. We begin by describing the platform AIDR (Artificial Intelligence for Disaster Response), which collects human annotations over time to create and maintain automatic supervised classifiers for social media messages. Next, we study two significant challenges in its design: (1) identifying which elements must be labeled by humans, and (2) determining when to ask for such annotations to be done. The first challenge is selecting the items to be labeled by crowdsourcing workers to maximize the productivity of their work. The second challenge is to schedule the work in order to reliably maintain high classification accuracy over time. We provide and validate answers to these challenges by extensive experimentation on realworld datasets.