CHS: Small: Supporting Crowdsourced Sensemaking in Big Data with Dynamic Context Slices

这项研究将探讨如何将众包和计算技术相结合，以支持从事复杂的意义制造任务的个人分析师的努力，例如识别对国家安全的威胁或确定照片中的人物和地点的名称。目前，这种复杂的任务超出了最先进的机器学习技术或众包工作流程的能力，即使是训练有素的专家也很难完成这些任务。 现在网上有大量的数据，但理解它们是一项挑战，因为人类的认知虽然非常强大，但资源有限。可视化分析工具试图通过利用信息可视化和数据挖掘的互补优势来克服这一限制，但这些工具通常有助于低级别任务，需要用户付出大量努力。众包已经成为一种很有前途的技术，可以将人类智能应用于计算机无法轻松解决的问题，但要让群体帮助个人完成复杂的意义构建任务，必须解决两个重大挑战。首先，我们必须理解在意义构建循环的每个阶段，群体与计算在什么时候更有用。其次，我们必须克服大多数人群工作者有限的时间和专业知识，以维持深入，复杂的调查路线。首先，它将进行一项实验室研究，让个人执行复杂的意义构建任务，以了解他们用来做决策的上下文的类型和数量，以及意义构建循环如何分解为子任务。其次，它将针对每一个最有前途的意义构建子任务进行一系列实验，将众包与自动化技术进行比较。第三，它将尝试不同的人群工作流程，以开发一个经过修改的意义构建循环，针对人群和计算的相对优势进行优化，并基于这种方法开发一个软件原型。软件设计的核心是“上下文切片”的新概念，这是一种创新技术，通过仅向人群工作人员提供完成指定任务所需的信息来解决人群工作人员的短暂性，从而允许多个工作人员进行复杂的调查。第四项实验将通过比较软件的性能与第一项研究中建立的基线来评估这种方法。

This research will investigate how crowdsourcing and computational techniques can be combined to support the efforts of an individual analyst engaged in a complex sensemaking task, such as identifying a threat to national security or determining the names of people and places in a photograph. Currently, such complex tasks are beyond the capabilities of the most advanced machine learning techniques or crowdsourcing workflows, and even trained experts struggle to perform them. Huge quantities of data are now available online, but making sense of them is challenging because human cognition, while remarkably powerful, is nevertheless a limited resource. Visual analytics tools seek to overcome this limitation by leveraging the complementary strengths of information visualization and data mining, but these tools generally assist with low-level tasks, requiring significant effort on the part of users. Crowdsourcing has emerged as a promising technique for applying human intelligence to problems computers cannot easily solve, but for crowds to assist individuals with complex sensemaking tasks, two significant challenges must be addressed. First, we must understand when crowds versus computation are more useful at each phase in the sensemaking loop. Second, we must overcome the limited time and expertise of most crowd workers to sustain deep, complex lines of inquiry.This research addresses both of these challenges through a series of four experiments. First, it will conduct a laboratory study where individuals perform complex sensemaking tasks to understand what types and amounts of context they use to make decisions, and how the sensemaking loop might be decomposed into subtasks. Second, it will conduct a series of experiments comparing crowdsourcing to automated techniques for each of the most promising sensemaking subtasks. Third, it will experiment with different crowd workflows to develop a revised sensemaking loop, optimized for the relative strengths of crowds and computation, and develop a software prototype based on this approach. At the core of the software design is the novel concept of "context slices," an innovative technique for addressing the transience of crowd workers by giving them only the information they need to complete their assigned task, allowing complex investigations to be pursued across multiple workers. The fourth experiment will evaluate this approach by comparing performance with the software to the baselines established in the first study.