Streamlining Embedded Assessment to Understand Citizen Scientists' Skill Gains

作为加强非正式环境中学习的总体战略的一部分，推进非正式STEM学习（AISL）项目资助了用于各种环境的创新研究、方法和资源。该项目将协同设计、测试和研究有效和高效的方法，以开发公民科学志愿者科学探究技能的嵌入式评估（EAs），以便更好地了解这些公民科学经验对志愿者科学探究能力的影响。ea是整合到学习经验中的评估活动，允许学习者以一种不引人注目的方式展示他们的能力。获得科学探究技能是公民科学体验的基本特征，甚至是决定性特征，它对数据质量有直接影响。评估CS对志愿者科学探究技能直接影响的方法有限。该项目将产生供不同CS项目使用的EA度量，CS项目可用于开发EA评估工具的策略，以及记录该创新方法在一系列CS环境中的机会、支持和障碍的研究结果。调查结果和初步资源将通过会议、研讨会、同行评审出版物、社区网站和其他相关场所与广泛的CS利益相关者分享。这项工作的结果也有可能推广到其他非正式的科学学习经验，使公众参与科学。该项目将解决两个研究问题：(1)哪些过程对开发广泛适用的环境评估方法或措施有用？(2)当公民科学组织使用EA时，我们可以了解到志愿者科学探究技能的收获？这些问题将通过以设计为基础的研究来解决，研究重点是两种精简策略。为了重新构建数据验证策略，来自五个已建立的公民科学项目的六位领导者将对他们现有的数据库进行二次分析，以揭示CS志愿者目前未在数据中探索的技能收益。对于公共度量策略，十个CS项目将协作创建并测试选择的基于识别的技能的公共EA度量。数据将通过会议记录、参与者访谈和行动计划以及志愿者技能所得来收集，以捕捉每种战略的过程和成果。数据将分析使用接地理论，多过程技术，多层次模型，和方差的重复测量分析。基于设计的研究框架将通过启动参与CS项目的评估，并为两种不同的EA策略提供初始资源，从而显著扩大项目影响，这两种策略有可能极大地改变实践和影响公民科学努力，最终使更多的人通过为科学努力做出贡献来学习。该项目将直接为15个参与的公民科学项目提供真实的绩效工具，以评估其项目的质量，这将扩大他们对CS志愿者技能的理解，并帮助他们更好地招募和支持不同的受众（包括农村、低收入和部落社区）。

As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches and resources for use in a variety of settings. The project will collaboratively design, test and study effective and efficient ways to develop embedded assessments (EAs) of citizen science (CS) volunteer scientific inquiry skills in order to better understand the impact of these CS experiences on volunteer scientific inquiry abilities. EAs are assessment activities that are integrated into the learning experience and allow learners to demonstrate their competencies in an unobtrusive way. The acquisition of scientific inquiry skills is an essential, even defining, characteristic of citizen science experiences that has a direct influence on data quality. Methods for assessing the direct impact of CS on volunteers' scientific inquiry skills are limited. The project will result in EA measures designed for use by diverse CS projects, strategies that CS projects can use to develop EA assessment tools, and research findings that document opportunities, supports and barriers of this innovative method across a range of CS contexts. Findings and initial resources will be shared with the broad array of stakeholders in CS through conferences, workshops, peer-reviewed publication, community websites and other relevant venues. The results of this work also have the potential to generalize to other informal science learning experiences that engage the public in scienceThe project will address two research questions: (1) What processes are useful for developing broadly applicable EA methods or measures? and (2) What can we learn about gains in volunteers' scientific inquiry skills when citizen science organizations use EA? These will be addressed through design-based research focused on two streamlining strategies. For the reframing data validation strategy, six leaders from five established citizen science projects will conduct secondary analyses of their existing databases to uncover the skill gains of CS volunteers that are currently unexplored in their data. For the common measure strategy, ten CS projects will collaborate to create and test common EA measures of select identification-based skills. Data will be gathered through meeting notes, participant interviews and action plans, and volunteer skill gains to capture process and products of each strategy. Data will be analyzed using grounded theory, multiple process techniques, multilevel models, and repeated-measures analysis of variance. The design-based-research framework will significantly expand project impacts by jump-starting evaluation of the participating CS projects and by producing initial resources for two distinct EA strategies that have the potential to dramatically alter practice and impact citizen science efforts to ultimately enable more people to learn by contributing to the science endeavor. The project will directly equip the 15 participating citizen-science projects with authentic performance tools to assess the quality of their programing, which will expand their understanding of CS volunteer skills and help them better recruit and support their varied audiences (including rural, low-income and tribal communities).