EAGER: Collaborative Research: A Computational Model for Evaluating the Quality of Citizen Science Contributions

公民科学是一种研究合作形式，让公众参与科学项目，为解决问题和社区参与带来多种声音和想法。公民科学可能是强大的，因为虽然特定的个人可能缺乏正式的专业知识，他们贡献高质量数据和新方向的能力有限，但一群人可能共同拥有识别和解决难题所需的专业知识和创造力。然而，从人群中收集科学数据的一个主要问题是贡献的数据的不同质量及其与科学假设的相关性。PIS将探索从计算创造力研究中得出衡量标准的潜力，以自动评估公民科学数据的质量，作为对现有人类对数据质量评估研究的补充。该项目还将探索如何将自动质量评估纳入向人群中的个人提出关于数据质量的建议的代理中，从而产生一个计算代理的原型，该计算代理衡量文明科学贡献的新颖性和价值。该项目将为未来在计算代理方面的研究提供信息，这些计算代理向大众学习并为人群做出贡献，以应对与公民科学众包的数据和想法的质量相关的挑战。更具体地说，该项目包括a)基于好的贡献不仅是可靠和准确的，而且是新颖和令人惊讶的概念，开发一个公民科学数据质量模型；b)根据被人类贴上质量标签的公民科学数据对该模型进行评估；以及c)关于计算质量反馈对人群成员的行为和感知的影响的初步研究。将质量评估扩展到包括创造力，并能够自动进行此类评估，对公民科学项目具有潜在的变革作用。PIS将在公民科学项目中展示其以代理人为基础的质量模式，包括他们自己的NatureNet项目，该项目让人群参与自然保护区的数据收集，并参与设计科学挑战和促进数据收集的互动体验。

Citizen science is a form of research collaboration that involves members of the public in scientific projects, bringing multiple voices and ideas to problem solving and community participation. Citizen science can be powerful because while specific individuals may lack formal expertise and be limited in their ability to contribute high-quality data and new directions, a crowd of individuals may collectively possess the expertise and creativity necessary for identifying and solving difficult problems. However, a major concern for collecting scientific data from the crowd is the varied quality of the contributed data and their relevance to scientific hypotheses. The PIs will explore the potential for deriving metrics from research on computational creativity to automatically assess the quality of citizen science data as a complement to existing research on human assessment of data quality. The project will also explore how the automated assessment of quality can be incorporated into an agent that makes suggestions to individuals in the crowd about the quality of their data, resulting in a prototype for a computational agent that measures the novelty and value of a cizen science contribution. This project will inform future research in computational agents that learn from and contribute to the crowd in order to address challenges associated with the quality of the data and ideas from crowdsourcing in citizen science. More specifically, the project includes a) development of a model of citizen-science-data quality based on the notion that good contributions are not just reliable and accurate but also novel and surprising; b) an evaluation of the model against citizen-science data that has been labeled by humans for quality; and c) initial studies of the effect of computational quality feedback on the behavior and perception of members of the crowd. Extending quality assessment to include creativity and being able to make such assessments automatically is potentially tranformative for citizen science projects. The PIs will demonstrate their agent-based model of quality in citizen science projects including their own NatureNet project, which involves crowd participants in data collection in nature preserves and also in the design of scientific challenges and interaction experience that facilitate data collection.