Streamlining Embedded Assessment to Understand Citizen Scientists' Skill Gains

作为加强非正式环境中学习的整体战略的一部分，推进非正式STEM学习（AISL）计划资助创新研究，方法和资源，用于各种环境。该项目将合作设计，测试和研究有效和高效的方法来开发嵌入式评估（EA）的公民科学（CS）志愿者科学探究技能，以更好地了解这些CS经验对志愿者科学探究能力的影响。EA是整合到学习体验中的评估活动，允许学习者以不引人注目的方式展示他们的能力。科学探究技能的获得是公民科学经验的一个基本特征，甚至是决定性特征，对数据质量有直接影响。用于评估CS对志愿者科学探究技能的直接影响的方法是有限的。该项目将导致EA的措施，旨在使用不同的CS项目，战略CS项目可以用来开发EA评估工具，和研究结果，记录的机会，支持和障碍，这种创新的方法在一系列的CS上下文。将通过会议、讲习班、同行评审的出版物、社区网站和其他相关场所，与CS的广泛利益攸关方分享调查结果和初步资源。这项工作的结果也有可能推广到其他非正式的科学学习经验，使公众参与科学该项目将解决两个研究问题：（1）什么样的过程是有用的发展广泛适用的EA方法或措施？以及（2）当公民科学组织使用EA时，我们可以从志愿者的科学探究技能中获得什么？这些问题将通过基于设计的研究来解决，重点是两个精简战略。对于重构数据验证策略，来自五个已建立的公民科学项目的六位领导者将对其现有数据库进行二次分析，以揭示目前未在其数据中探索的CS志愿者的技能增益。对于共同的测量策略，十个CS项目将合作创建和测试选择识别为基础的技能的共同EA措施。将通过会议记录、参与者访谈和行动计划以及志愿人员技能增益收集数据，以掌握每项战略的过程和产品。将使用扎根理论、多处理技术、多水平模型和重复测量方差分析对数据进行分析。设计为基础的研究框架将显着扩大项目的影响，通过启动参与CS项目的评估，并通过为两个不同的EA战略产生初始资源，这些战略有可能极大地改变实践并影响公民科学的努力，最终使更多的人能够通过为科学奋进做出贡献来学习。该项目将直接为15个参与的公民科学项目提供真实的绩效工具，以评估其计划的质量，这将扩大他们对CS志愿者技能的理解，并帮助他们更好地招募和支持他们的各种受众（包括农村，低收入和部落社区）。

As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches and resources for use in a variety of settings. The project will collaboratively design, test and study effective and efficient ways to develop embedded assessments (EAs) of citizen science (CS) volunteer scientific inquiry skills in order to better understand the impact of these CS experiences on volunteer scientific inquiry abilities. EAs are assessment activities that are integrated into the learning experience and allow learners to demonstrate their competencies in an unobtrusive way. The acquisition of scientific inquiry skills is an essential, even defining, characteristic of citizen science experiences that has a direct influence on data quality. Methods for assessing the direct impact of CS on volunteers' scientific inquiry skills are limited. The project will result in EA measures designed for use by diverse CS projects, strategies that CS projects can use to develop EA assessment tools, and research findings that document opportunities, supports and barriers of this innovative method across a range of CS contexts. Findings and initial resources will be shared with the broad array of stakeholders in CS through conferences, workshops, peer-reviewed publication, community websites and other relevant venues. The results of this work also have the potential to generalize to other informal science learning experiences that engage the public in scienceThe project will address two research questions: (1) What processes are useful for developing broadly applicable EA methods or measures? and (2) What can we learn about gains in volunteers' scientific inquiry skills when citizen science organizations use EA? These will be addressed through design-based research focused on two streamlining strategies. For the reframing data validation strategy, six leaders from five established citizen science projects will conduct secondary analyses of their existing databases to uncover the skill gains of CS volunteers that are currently unexplored in their data. For the common measure strategy, ten CS projects will collaborate to create and test common EA measures of select identification-based skills. Data will be gathered through meeting notes, participant interviews and action plans, and volunteer skill gains to capture process and products of each strategy. Data will be analyzed using grounded theory, multiple process techniques, multilevel models, and repeated-measures analysis of variance. The design-based-research framework will significantly expand project impacts by jump-starting evaluation of the participating CS projects and by producing initial resources for two distinct EA strategies that have the potential to dramatically alter practice and impact citizen science efforts to ultimately enable more people to learn by contributing to the science endeavor. The project will directly equip the 15 participating citizen-science projects with authentic performance tools to assess the quality of their programing, which will expand their understanding of CS volunteer skills and help them better recruit and support their varied audiences (including rural, low-income and tribal communities).