CHS: Small: Collaborative Research: Optimizing the Human-Machine System for Citizen Science

本研究旨在提高支持公民科学的在线系统的效率、准确性和可用性，在这些系统中，围绕严肃的科学研究项目组织的社区结合了业余爱好者和专业人士的贡献。为了最有效地响应跨多个领域日益增长的数据洪流，公民科学平台需要更加动态和复杂-结合智能任务分配和机器学习策略。同时利用人类和机器智能的系统引起了各个学科的科学家的兴趣。无论是将其视为社交机器还是主动学习系统，这些混合系统都表现出复杂的行为，需要对其进行有效的系统设计。例如，机器学习研究人员专注于使用公民科学项目产生的大型训练集来训练算法，这些算法随后应用于完整的数据集。然而，这种串行处理可能不是最有效地利用人类或机器的努力。该项目的主要研究目标是调查组合人机系统的整体效率如何受到单独组件及其相关属性的影响，以及对人类或机器分类器或两者的影响。这个过程将测试一个假设，即提高整体效率实际上会减少专家人类分类器的负载，而不是像目前所要求的那样，需要更大的机器专家训练集。该项目将研究人类和机器分类器的动态组合，首次获得如何在真实、灵活的公民科学平台上最佳地共享负载的知识。这项研究工作将通过在现有的Zooniverse基础设施（世界领先的在线公民科学平台）上构建和部署软件模块来支持。它将(1)进行有效和动态的任务分配，近实时地区分有经验和没有经验的分类器，以及熟练和不熟练的分类器；(2)将人与机器分类动态结合，在志愿者提供的训练数据量不断增加的基础上，定期训练自动分类例程。然后，这种新软件将用于一种新的“级联过滤”模式，将复杂的分类问题简化为一系列单一的二进制任务。该项目开发的软件将为希望利用新基础设施的领域科学家和社会机器研究人员提供一套完全灵活的功能，以满足他们特定问题的需求。

This research aims to improve the efficiency, accuracy, and usability of online systems supporting citizen science, in which communities organized around serious scientific research projects combine the contributions of amateurs and professionals. In order to respond most efficiently to the increasing data deluge across multiple domains, citizen science platforms need to be more dynamic and complex - incorporating intelligent task assignment and machine learning strategies. Systems that make use of both human and machine intelligence are of interest to scientists from a wide range of disciplines. Whether viewed as social machines or as active learning systems in which progressive input from humans improves machine learning, these hybrid systems exhibit complex behavior which needs to be understood for effective system design. For example, machine learning researchers have concentrated on using the large training sets produced by citizen science projects in order to train algorithms that are later applied to a full dataset. Yet this serial processing may not be the most efficient use of the human or machine effort. The main research goal of this project is to investigate how the overall efficiency of the combined human-machine system is impacted by the separate components and their related properties and what the implications are for either human or machine classifiers or both. This process will test the hypothesis that improved overall efficiency will actually reduce the load on expert human classifiers instead of, as currently required, needing larger expert training sets for machines. This project will investigate the dynamic combination of human and machine classifiers, gaining for the first time knowledge of how load can be optimally shared in a real, flexible citizen science platform. This research effort will be supported by building and deploying software modules on the existing Zooniverse infrastructure, the world-leading platform for online citizen science. It will (1) carry out efficient and dynamic task assignment, distinguishing in near-real time between experienced and inexperienced, and between skilled and less skilled classifiers; and (2) combine human and machine classifications dynamically, periodically training automatic classification routines on the increasing volume of training data produced by volunteers. This new software will then be utilized in a novel "cascade filtering" mode that reduces complex classification problems into a series of single binary tasks. The software developed in this project will provide domain scientists and social machine researchers who wish to exploit the new infrastructure with a fully flexible suite of functions appropriate to the needs defined by their specific problems.