Collaborative Research: Designing Assessments in Physical Science Across Three Dimensions

这是伊利诺斯大学芝加哥分校、密歇根州立大学和SRI国际之间的一项合作提案，旨在为中学（6-8年级）物理科学开发、测试和分析一系列技术支持的诊断性课堂评估。评估与K-12科学教育框架中建议的绩效评估和以证据为中心的设计方法一致（NRC, 2012）。这项研究的重点是考虑到科学内容和实践的相互依存关系的新的学习方法的发展。两个学科核心思想——物质及其相互作用和能量——以及两个科学和工程实践——构建解释和设计解决方案，以及开发和使用模型——用于此目的。研究的问题是：(1)系统应用循证设计（ECD）评价过程的科学评价特征是什么？(2)评估设计在多大程度上可以将关键核心思想、横切概念和科学/工程实践维度作为设计架构的一部分，以分离和整合这些维度的方式纳入？(3)有什么证据表明科学学习的多个维度（如内容、实践和横切概念）在回应这些评估的学生的表现中是可分离和可恢复的？(4)这些评估的教学敏感性如何？（即，他们是否对学生以符合NRC框架所包含的愿景的方式学习科学的机会表现出不同的和适当的敏感性？）；(5)使用考虑到这些新评估的有效性论证的解释和证据组成部分的多方面有效性框架，可以为这些评估的有效性提供哪些形式的证据？(6)相对于形成性评估过程而言，最能满足课堂教师需求的评估有哪些特点？这些评估和评分过程需要以何种方式设计，以支持教师有效实施？(7)在设计和实施集中于合并内容和实践绩效期望的评估时，技术提供的独特的支持和机会是什么？评估在佛罗里达州的三个学区和一个实验学校以及威斯康星州的一个学区使用“通过科学技术调查和质疑我们的世界”课程进行迭代设计和管理。佛罗里达州的三个学区都有使用典型课程的教室。这些评估也将由这些教室里的学生进行管理和测试。为了解决这些研究问题，该项目开展了五项主要任务：(1)利用ECD流程开发评估项目，以记录和指导项目的一致性；(2)进行一致性研究，检讨设计模式和任务模板；(3)一项认知分析研究，以实证调查项目在多大程度上引出了预期的指导方针；(4)三个实证研究，包括：(a)一年级教师（n=6）和学生（n=180）的早期测试，(b)二年级教师（n=12）和学生（n=360）的试点测试，(c)三年级教师（n=30）和学生（n=900）的主要研究；(5)利用教师焦点小组的反馈和对以往研究中学生表现数据的分析，调查评估项目的形成性使用。项目成果是：(a)基于研究和实地测试的评估原型，用于衡量学生围绕两个物理科学核心思想和两个科学与工程实践的思考；(b)研究中使用的相关数据和程序；(c)形成性使用评估的框架，包括指导方针、评分标准和评估设计决策的标准。

This is a collaborative proposal among the University of Illinois at Chicago, Michigan State University, and SRI International to develop, test, and analyze sets of technology-supported diagnostic classroom assessments for middle school (grades 6-8) physical science. Assessments are aligned with the performance assessment and evidence-centered design methodologies suggested in the Framework for K-12 Science Education (NRC, 2012). The study focuses on the development of new measures of learning that take into account the interdependence of science content and practice. Two disciplinary core ideas--Matter and its Interactions, and Energy--and two scientific and engineering practices--Constructing Explanations and Designing Solutions, and Developing and Using Models--are used for this purpose. The research questions are: (1) What are the characteristic features of science assessments based upon systematic application of the Evidence-Centered Design (ECD) assessment process?; (2) To what extent can assessment designs incorporate critical core idea, crosscutting concept and science/engineering practice dimensions in ways that both separate and integrate these dimensions as part of the design architecture?; (3) What is the evidence that the multiple dimensions of science learning (e.g., content, practices and crosscutting concepts) are separable and recoverable in the performance of students who respond to these assessments?; (4) How instructionally sensitive are these assessments? (i.e., Do they show differential and appropriate sensitivity to students' opportunity to learn science in ways consistent with the vision contained in the NRC Framework?); (5) What forms of evidence can be provided for the validity of these assessments using a multifaceted validity framework that takes into account both the interpretive and evidentiary components of a validity argument for these new assessments?; (6) What are the characteristics of assessments that best serve the needs of classroom teachers relative to a formative assessment process and in what ways do such assessments and scoring processes need to be designed to support effective teacher implementation?; and (7) What are the unique affordances and opportunities provided by technology in designing and implementing assessments focused on merging content & practices performance expectations? Assessments are iteratively designed and administered in three school districts and a laboratory school in Florida and one school district in Wisconsin using the "Investigating and Questioning our World through Science and Technology" curriculum. The three school districts in Florida have classrooms that are using typical curriculum. The assessments will also be administered and tested with students in these classrooms. To address the research questions, the project conducts five major tasks: (1) development of assessment items using the ECD process to document and guide coherence of items; (2) an alignment study to review design patterns and task templates; (3) a cognitive analysis study to empirically investigate the extent to which the items elicit the intended guidelines; (4) three empirical studies, including (a) an early-stage testing with teachers (n=6) and students (n=180) in Year 1, (b) a pilot testing in Year 2 with teachers (n=12) and students (n=360), and (c) a main study in Year 3 with teachers (n=30) and students (n=900); and (5) a study to investigate the formative use of the assessment items using teacher focus groups' feedback and analysis of student performance data from previous studies. Project outcomes are: (a) research-informed and field-tested assessment prototypes that measure students' thinking around the two physical science core ideas and the two scientific and engineering practices; (b) relevant data and procedures used in the studies; and (c) a framework for the formative use of the assessments, including guidelines, scoring rubrics, and criteria for assessment design decisions.