SimScientists Games: Development of Simulation-Based Game Designs to Enhance Formative Assessment and Deep Science Learning in Middle School

探索研究K-12(DRK-12)计划旨在通过研究和开发创新资源、模型和工具，显著提高K-12学前班学生和教师在科学、技术、工程和数学(STEM)方面的学习和教学。DRK-12计划中的项目建立在STEM教育的基础研究和先前的研究和开发努力的基础上，这些研究和开发工作为拟议的项目提供了理论和经验证明。该项目是一项为期四年的设计和开发研究，提交给该计划的评估链。它将侧重于了解根据研究性学习和评估设计原则设计的教育游戏如何能够更好地评估和促进学生的科学知识、科学过程技能的应用以及学习动机和参与度。该项目将开发一种新的游戏类型，作为形成性评估资源，旨在收集游戏过程中科学学习的证据，以提示的形式提供反馈和指导，并加强中学(6-8年级)学生关于下一代科学标准(NGSS)(AQUE，2013)中描述的关于生态系统的生命科学概念和调查实践。这些游戏将建立在NSF之前资助的基于模拟的、嵌入课程的评估的设计之上，其中包括学生进度报告和反思活动，允许教师向学生提供反馈并调整教学。游戏的设计将借鉴多个研究方向，如认知，特别是基于模型的学习；原则性评估设计；以及动机。该项目旨在提供引人入胜的活动，以了解和研究生态系统的系统组成部分、角色、相互作用和种群动态，该项目将产生两套全面的游戏：(1)生物和相互作用，(2)紧急种群水平：管理生态系统。每一款游戏都将由逐步进阶的迷你游戏组成。加州湾区的24名中学教师将参与这项研究。教师专业发展(PD)将包括面对面的会议和一个允许参与者和促进者之间进行广泛互动的在线平台。PD将强调基于生态系统模拟的课程模块与其国家标准、教学材料和新游戏的一致性。该项目将解决六个研究问题：(1)游戏与NGSS中生态系统的横切概念、核心思想和探究实践的一致性如何？(2)游戏组件满足质量标准的程度如何？(3)游戏与现有的基于模拟的课程模块和教师现有的教学顺序的整合情况如何？(4)游戏的使用对学生对科学概念、科学实践和协作技能的理解有何影响？(5)游戏性的成功与外部结果测量(包括基于模拟的基准和前后测试)的表现如何相关？(6)游戏的使用如何影响学生对科学学习的投入？在一年的可用性研究中，该项目将测试、分析和修改游戏的阿尔法版本。在第二年，测试版的课堂可行性研究将为进一步的修订提供信息。在第三年，六名教师将对游戏进行试点测试。第四年的第二个试点测试将通过比较学生在使用现有基于模拟的课程嵌入评估和反思活动的课堂上的表现，以及使用课程嵌入评估和新游戏的课堂来检查游戏的有效性。数据收集和分析策略包括：(A)一致性审查；(B)焦点小组和可用性测试；(C)认知实验室的结构效度和可用性；(D)游戏报告(徽章)；(E)美国科学促进会项目的前后测试；(F)基准评估数据；(G)学生对游戏和科学的兴趣；(H)教师调查；(I)案例研究；(J)游戏质量分析；(K)差异项目功能；(L)协方差分析；以及(M)后测分数(结果变量)的方差分析，以比较学生群体的平均数(通过干预模式)和他们以前的科学成就水平。

The Discovery Research K-12 (DRK-12) program seeks to significantly enhance the learning and teaching of science, technology, engineering, and mathematics (STEM) by preK-12 students and teachers through research and development of innovative resources, models, and tools. Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects. This project is a four-year design and development study submitted to the assessment strand of the program. It will focus on understanding how educational games, designed according to research-based learning and assessment design principles, can better assess and promote students' science knowledge, application of science process skills, and motivation and engagement in learning. The project will develop a new genre of games to serve as formative assessment resources designed to collect evidence of science learning during gameplay, provide feedback and coaching in the form of hints, and reinforce middle grade (6th-8th) students' life science concepts and investigation practices about ecosystems described in the Next Generation Science Standards (NGSS) (Achieve, 2013). The games will build on the designs of the simulation-based, curriculum-embedded assessments developed in previous NSF-funded efforts, which include student progress reports and reflection activities that allow teachers to provide feedback to students and adjust instruction. The design of the games will draw from multiple lines of research, such as cognition, particularly model-based learning; principled assessment design; and motivation. Intended to provide engaging activities for understanding and investigating the system components, roles, interactions, and population dynamics of ecosystems, the project will produce two sets of comprehensive games: (1) Organisms and Interactions, and (2) Emergent Population Levels: Managing an Ecosystem. Each game will consist of progressively advanced mini-games. Twenty-four California Bay Area middle school teachers will participate in the study. Teacher professional development (PD) will include face-to-face sessions and an online platform that permits a wide range of interactions among participants and the facilitators. The PD will emphasize the alignment of the ecosystem simulation-based curriculum modules with their state standards, instructional materials, and the new games. The project will address six research questions: (1) How well do the games align with the ecosystem crosscutting concepts, core ideas, and inquiry practices in the NGSS?; (2) How well do game components meet quality standards?; (3) How well do the games integrate with the existing simulation-based curriculum modules and the teachers' existing instructional sequence?; (4) What effect does the use of the games have on students' understanding of the science concepts, scientific practices, and collaboration skills?; (5) How does success in gameplay relate to improved performance on the external outcome measures comprised of the simulation-based benchmark and the pre/posttest?; and (6) How does the use of the games affect students' engagement in science learning? In a Year 1 usability study, the project will test, analyze, and revise alpha versions of the games. In Year 2, a classroom feasibility study of beta versions will inform further revisions. In Year 3, six teachers will pilot-test the games. A second pilot test in Year 4 will examine the effectiveness of the games by comparing student performance in classes using the existing simulation-based curriculum-embedded assessments and reflection activities with classes using the curriculum-embedded assessments plus the new games. Data collection and analysis strategies include: (a) alignment reviews; (b) focus groups and usability testing; (c) cognitive labs for construct validity and usability; (d) game reports (badges); (e) pre/posttest of American Association for the Advancement of Science (AAAS) items; (f) benchmark assessment data; (g) student interest in the games and science; (h) teacher surveys; (i) case studies; (j) game quality analysis; (k) differential item functioning; (l) analysis of covariance; and (m) analysis of variance on posttest scores (outcome variable) to compare the means across student groups (by intervention mode) and their prior science achievement levels.