CHS: Small: Collaborative Research: Optimizing the Human-Machine System for Citizen Science

该项目研究用于分析大型复杂数据集的最佳人机协作。该项目利用机器学习的进步来开发新的公民科学系统基础设施，该基础设施将被构建到Zooniverse-一个大型的、开源的在线公民科学平台。公民科学是一种对大量数据进行分布式分析的成熟方法，在这种方法中，在线志愿者帮助完成需要人类模式识别的任务。一个例子是在银河动物园项目中识别星系的形态。规模更大的数据集正在迫在眉睫。设计一个人机系统来加速对已知类别的标记，同时解决检测有趣的异常(提出新的现象)的问题，需要回答几个关键的研究问题，即人类和机器如何最好地相互补充。由于该项目的新技术将被纳入Zooniverse，它们将可供所有人在许多学科的公民科学项目中使用。该项目的其他好处包括通过Zooniverse吸引200多万参与公民科学的公众，在明尼苏达大学计算机科学编码夏令营吸引年轻女性，并为数据科学硕士项目的学生提供为期一年的顶峰项目，让他们从事现实世界的研究，同时为他们在数据科学领域的职业生涯做好准备。该项目将对人和机器分类器之间的负载平衡进行详细的调查，针对给定任务的领域研究所需的速度、准确性、完整性或纯度进行优化。研究计划遵循两个主旨：(1)分类效率研究，以优化跨多个领域和任务类型的已知类别的分类效率；(2)系统化的偶然研究，以提高发现效率，包括检测罕见实例、不寻常的发现和新的类别。该项目开发了新的基础设施，该基础设施建立在Zooniverse公民科学平台现有能力的基础上，通过两个模块：(1)机器集成基础设施，以方便地整合和组合项目中的机器，以提高分类效率，并探索系统化的偶然发现的机器驱动的组成部分；(2)为志愿者提供升级策略，以实现对紧急课程的人驱动识别。最终，人和机器驱动的机制将结合在一起，形成一个组合的人机系统，将测试其识别未知、罕见或难以识别类别的能力。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project investigates optimal human-machine collaboration for analysis of large, complex data sets. The project uses advances in machine learning to develop new citizen science system infrastructure to be built into Zooniverse --- a large, open-source platform for online citizen science. Citizen Science is an established method for distributed analysis of large quantities of data in which online volunteers help with tasks requiring human pattern recognition. An example is identifying morphology of galaxies in the Galaxy Zoo project. Much larger data sets are looming on the horizon. Designing a human-machine system to accelerate labeling of known classes at the same time as solving the problem of detecting interesting anomalies (suggesting new phenomena) requires answering several crucial research questions about how humans and machines best complement one another. Since the project's new techniques will be incorporated into Zooniverse, they will be available to all for use in citizen science projects across many disciplines. Additional benefits of this project include engaging over 2 million members of the public who participate in citizen science through Zooniverse, engaging young women in University of Minnesota computer science coding camps, and providing year-long capstone projects for Data Science Masters program students to engage in real-world research while preparing them for careers in data science. This project will carry out a detailed investigation of load balancing between human and machine classifiers, optimizing for the speed, accuracy, completeness or purity required by the domain research for a given task. The research program follows two thrusts: (1) Classification Efficiency Studies to optimize the classification efficiency of known classes across multiple domains and task types; and (2) Systematized Serendipity Studies to increase the efficiency of discovery, including detection of rare instances, unusual findings, and new classes. The project develops new infrastructure that builds on the existing capabilities of the Zooniverse citizen science platform through two modules: (1) Machine Integration Infrastructure to readily incorporate and combine machines on projects to increase classification efficiency as well as explore the machine-driven component of systematized serendipity; and (2) Leveling-up Strategies for Volunteers to enable human-driven identification of emergent classes. Ultimately, the human- and machine-driven mechanisms will be joined to form a combined human-machine system that will be tested for its ability to identify unknown, rare, or difficult to identify classes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.