SimScientists Games: Development of Simulation-Based Game Designs to Enhance Formative Assessment and Deep Science Learning in Middle School

Discovery Research K-12（DRK-12）计划旨在通过创新资源、模型和工具的研究和开发，显着增强K-12预科学生和教师对科学、技术、工程和数学（STEM）的学习和教学。DRK-12计划中的项目建立在STEM教育的基础研究和先前的研究和开发工作的基础上，这些研究和开发工作为拟议项目提供了理论和经验依据。该项目是一个为期四年的设计和开发研究，提交给该计划的评估链。本课程将着重了解根据研究性学习和评估设计原则设计的教育游戏如何更好地评估和促进学生的科学知识，科学过程技能的应用，以及学习的动机和参与。该项目将开发一种新的游戏类型，作为形成性评估资源，旨在收集游戏过程中科学学习的证据，以提示的形式提供反馈和指导，并加强中年级（6 - 8年级）学生的生命科学概念和下一代科学标准（NGSS）中描述的生态系统调查实践（Achieve，2013）。这些游戏将建立在以前NSF资助的工作中开发的基于模拟的嵌入式评估的设计基础上，其中包括学生进度报告和反思活动，允许教师向学生提供反馈并调整教学。游戏的设计将借鉴多方面的研究，如认知，特别是基于模型的学习;原则性的评估设计;和动机。该项目旨在为理解和调查生态系统的系统组成部分，角色，相互作用和种群动态提供参与性活动，该项目将制作两套综合游戏：（1）有机体和相互作用，以及（2）新兴人口水平：管理生态系统。每个游戏将包括逐步先进的迷你游戏。24名加州湾区的中学教师将参与这项研究。教师专业发展（PD）将包括面对面的会议和一个在线平台，允许参与者和主持人之间进行广泛的互动。PD将强调基于生态系统模拟的课程模块与其国家标准，教学材料和新游戏的一致性。该项目将解决六个研究问题：（1）游戏与NGSS中的生态系统横切概念、核心思想和探究实践的一致性如何？(2)游戏组件符合质量标准的程度如何？(3)这些游戏与现有的模拟课程模块和教师现有的教学顺序结合得如何？(4)游戏的使用对学生理解科学概念、科学实践和协作技能有什么影响？(5)游戏性的成功与外部结果测量（包括基于模拟的基准测试和前/后测试）的改进表现有什么关系？游戏的使用如何影响学生的科学学习投入？在第一年的可用性研究中，该项目将测试，分析和修改游戏的alpha版本。在第二年，测试版的课堂可行性研究将为进一步的修订提供信息。在第三年，六名教师将试点测试游戏。第二次试点测试将在四年级进行，通过比较学生在课堂上使用现有的基于模拟的嵌入式评估和反思活动与使用嵌入式评估加上新游戏的课堂上的表现，来检查游戏的有效性。数据收集和分析战略包括：（a）调整审查;（B）焦点小组和可用性测试;（c）结构有效性和可用性认知实验室;（d）游戏报告（徽章）;（e）美国科学促进协会项目的前/后测试;（f）基准评估数据;（g）学生对游戏和科学的兴趣;（h）教师调查;（i）案例研究;（j）游戏质量分析;（k）项目功能差异分析;（l）协方差分析;以及（m）后测分数的方差分析（结果变量），以比较学生群体的平均值（按干预模式）及其先前的科学成绩水平。

The Discovery Research K-12 (DRK-12) program seeks to significantly enhance the learning and teaching of science, technology, engineering, and mathematics (STEM) by preK-12 students and teachers through research and development of innovative resources, models, and tools. Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects. This project is a four-year design and development study submitted to the assessment strand of the program. It will focus on understanding how educational games, designed according to research-based learning and assessment design principles, can better assess and promote students' science knowledge, application of science process skills, and motivation and engagement in learning. The project will develop a new genre of games to serve as formative assessment resources designed to collect evidence of science learning during gameplay, provide feedback and coaching in the form of hints, and reinforce middle grade (6th-8th) students' life science concepts and investigation practices about ecosystems described in the Next Generation Science Standards (NGSS) (Achieve, 2013). The games will build on the designs of the simulation-based, curriculum-embedded assessments developed in previous NSF-funded efforts, which include student progress reports and reflection activities that allow teachers to provide feedback to students and adjust instruction. The design of the games will draw from multiple lines of research, such as cognition, particularly model-based learning; principled assessment design; and motivation. Intended to provide engaging activities for understanding and investigating the system components, roles, interactions, and population dynamics of ecosystems, the project will produce two sets of comprehensive games: (1) Organisms and Interactions, and (2) Emergent Population Levels: Managing an Ecosystem. Each game will consist of progressively advanced mini-games. Twenty-four California Bay Area middle school teachers will participate in the study. Teacher professional development (PD) will include face-to-face sessions and an online platform that permits a wide range of interactions among participants and the facilitators. The PD will emphasize the alignment of the ecosystem simulation-based curriculum modules with their state standards, instructional materials, and the new games. The project will address six research questions: (1) How well do the games align with the ecosystem crosscutting concepts, core ideas, and inquiry practices in the NGSS?; (2) How well do game components meet quality standards?; (3) How well do the games integrate with the existing simulation-based curriculum modules and the teachers' existing instructional sequence?; (4) What effect does the use of the games have on students' understanding of the science concepts, scientific practices, and collaboration skills?; (5) How does success in gameplay relate to improved performance on the external outcome measures comprised of the simulation-based benchmark and the pre/posttest?; and (6) How does the use of the games affect students' engagement in science learning? In a Year 1 usability study, the project will test, analyze, and revise alpha versions of the games. In Year 2, a classroom feasibility study of beta versions will inform further revisions. In Year 3, six teachers will pilot-test the games. A second pilot test in Year 4 will examine the effectiveness of the games by comparing student performance in classes using the existing simulation-based curriculum-embedded assessments and reflection activities with classes using the curriculum-embedded assessments plus the new games. Data collection and analysis strategies include: (a) alignment reviews; (b) focus groups and usability testing; (c) cognitive labs for construct validity and usability; (d) game reports (badges); (e) pre/posttest of American Association for the Advancement of Science (AAAS) items; (f) benchmark assessment data; (g) student interest in the games and science; (h) teacher surveys; (i) case studies; (j) game quality analysis; (k) differential item functioning; (l) analysis of covariance; and (m) analysis of variance on posttest scores (outcome variable) to compare the means across student groups (by intervention mode) and their prior science achievement levels.