CHS: Small: Collaborative Research: Optimizing the Human-Machine System for Citizen Science

CHS：小型：协作研究：优化公民科学的人机系统

• 批准号: 1619177
• 负责人: Lucy Fortson
• 金额: $ 35.96万
• 依托单位: University of Minnesota-Twin Cities
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-07-01 至 2019-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1619177&HistoricalAwards=false
• 关键词: CHS; Small; Collaborative; Research; Optimizing

This research aims to improve the efficiency, accuracy, and usability of online systems supporting citizen science, in which communities organized around serious scientific research projects combine the contributions of amateurs and professionals. In order to respond most efficiently to the increasing data deluge across multiple domains, citizen science platforms need to be more dynamic and complex - incorporating intelligent task assignment and machine learning strategies. Systems that make use of both human and machine intelligence are of interest to scientists from a wide range of disciplines. Whether viewed as social machines or as active learning systems in which progressive input from humans improves machine learning, these hybrid systems exhibit complex behavior which needs to be understood for effective system design. For example, machine learning researchers have concentrated on using the large training sets produced by citizen science projects in order to train algorithms that are later applied to a full dataset. Yet this serial processing may not be the most efficient use of the human or machine effort. The main research goal of this project is to investigate how the overall efficiency of the combined human-machine system is impacted by the separate components and their related properties and what the implications are for either human or machine classifiers or both. This process will test the hypothesis that improved overall efficiency will actually reduce the load on expert human classifiers instead of, as currently required, needing larger expert training sets for machines. This project will investigate the dynamic combination of human and machine classifiers, gaining for the first time knowledge of how load can be optimally shared in a real, flexible citizen science platform. This research effort will be supported by building and deploying software modules on the existing Zooniverse infrastructure, the world-leading platform for online citizen science. It will (1) carry out efficient and dynamic task assignment, distinguishing in near-real time between experienced and inexperienced, and between skilled and less skilled classifiers; and (2) combine human and machine classifications dynamically, periodically training automatic classification routines on the increasing volume of training data produced by volunteers. This new software will then be utilized in a novel "cascade filtering" mode that reduces complex classification problems into a series of single binary tasks. The software developed in this project will provide domain scientists and social machine researchers who wish to exploit the new infrastructure with a fully flexible suite of functions appropriate to the needs defined by their specific problems.

这项研究旨在提高支持公民科学的在线系统的效率、准确性和可用性，在这些系统中，围绕严肃的科研项目组织的社区结合了业余爱好者和专业人士的贡献。为了最有效地应对跨多个领域的日益增长的数据洪流，公民科学平台需要更加动态和复杂-纳入智能任务分配和机器学习策略。同时利用人类和机器智能的系统引起了来自广泛学科的科学家的兴趣。无论是被视为社交机器，还是被视为人类渐进输入改进机器学习的主动学习系统，这些混合系统都表现出复杂的行为，需要理解这些行为才能进行有效的系统设计。例如，机器学习研究人员一直专注于使用公民科学项目产生的大型训练集来训练算法，这些算法后来应用于完整的数据集。然而，这种连续处理可能不是对人类或机器努力的最有效利用。这个项目的主要研究目标是调查单独的组件及其相关属性如何影响组合式人机系统的整体效率，以及这对人或机器分类器或两者都有什么影响。这一过程将检验这样一种假设，即提高整体效率实际上将减少专家人工分类器的负载，而不是像目前所要求的那样，需要更大的机器专家训练集。这个项目将研究人和机器分类器的动态组合，首次获得关于如何在一个真实、灵活的公民科学平台中优化分担负载的知识。这项研究工作将通过在现有的Zooniverse基础设施上构建和部署软件模块来支持，Zooniverse基础设施是世界领先的在线公民科学平台。它将(1)进行有效和动态的任务分配，近乎实时地区分有经验和无经验的分类员，以及熟练和不熟练的分类员；(2)将人和机器分类动态结合，根据志愿者产生的越来越多的训练数据定期培训自动分类例程。然后，这个新的软件将被用于一种新的“级联过滤”模式，该模式将复杂的分类问题简化为一系列单一的二进制任务。在该项目中开发的软件将为希望开发新基础设施的领域科学家和社会机器研究人员提供一套完全灵活的功能，适合于他们特定问题定义的需求。

项目成果

Integrating human and machine intelligence in galaxy morphology classification tasks

• DOI: 10.1093/mnras/sty503
• 发表时间: 2018-06-01
• 期刊: MONTHLY NOTICES OF THE ROYAL ASTRONOMICAL SOCIETY
• 影响因子: 4.8
• 作者: Beck, Melanie R.;Scarlata, Claudia;Wright, Darryl
• 通讯作者: Wright, Darryl

A transient search using combined human and machine classifications

• DOI: 10.1093/mnras/stx1812
• 发表时间: 2017-07
• 期刊: Monthly Notices of the Royal Astronomical Society
• 影响因子: 4.8
• 作者: D. Wright;C. Lintott;S. Smartt;Kenneth W. Smith;L. Fortson;L. Trouille;Campbell Allen;Melanie Beck;Mark C. Bouslog;Amy Boyer;K. Chambers;H. Flewelling;Will Granger;E. Magnier;Adam McMaster;G. Miller;J. O’Donnell;Helen Spiers;J. Tonry;Marten Veldthuis;R. Wainscoat;C. Waters;M. Willman;Zach Wolfenbarger;D. O. D. O. Physics-D.-O.;University of Oxford Astrophysics Research Centre;S. O. Mathematics;Physics;Queen's University Belfast Minnesota Institute for Astrophysics-Queen's-University-Belfast-Minnesota-Institute-for-1422183721;U. D. O. Physics;Astronomy;University of Minnesota Center for Interdisciplinary Exploration-University-of-Minnesota-Center-for-Exploration-1422184500;Research in Astrophysics;D. Physics;Northwestern University Citizen Science Department;The Netherlands Institute for Radio Astronomy;U. Hawaii