Collaborative Research: Designing Assessments in Physical Science Across Three Dimensions

这是伊利诺伊大学芝加哥分校、密歇根州立大学和SRI国际为中学(6-8年级)物理科学开发、测试和分析一套由技术支持的诊断性课堂评估的合作提案。评估与《K-12科学教育框架》(NRC，2012)中建议的绩效评估和以证据为中心的设计方法保持一致。这项研究的重点是制定新的学习措施，考虑到科学内容和实践之间的相互依存。两个学科核心概念--物质及其相互作用和能量--以及两个科学和工程实践--构建解释和设计解决方案，以及开发和使用模型--被用于这一目的。研究问题是：(1)基于循证设计(ECD)评估过程的系统应用的科学评估的特征是什么？(2)评估设计可以在多大程度上结合关键核心理念、横切概念和科学/工程实践维度，将这些维度分离和整合为设计架构的一部分？(3)有什么证据表明，科学学习的多个维度(例如，内容、实践和横切概念)在回应这些评估的学生的表现中是可分离和可恢复的？(4)这些评估在教学上的敏感性如何？(例如，它们是否对学生以与NRC框架中包含的愿景一致的方式学习科学的机会表现出不同和适当的敏感性？)；(5)使用多方面的有效性框架，考虑到这些新评估的有效性论证的解释性和证据性部分，可以提供哪些形式的证据来证明这些评估的有效性？(6)相对于形成性评估过程，评估的特征是什么，最能满足课堂教师需要的是什么，需要以什么方式设计这样的评估和评分过程，以支持教师的有效实施？以及(7)在设计和实施侧重于合并内容和实践绩效预期的评估时，技术提供了哪些独特的启示和机会？在佛罗里达州的三个学区和一所实验室学校，以及威斯康星州的一个学区，使用“通过科学和技术调查和质疑我们的世界”课程，反复设计和管理评估。佛罗里达州的三个学区都有使用典型课程的教室。评估也将在这些教室里与学生一起进行管理和测试。为了解决研究问题，该项目进行了五项主要任务：(1)使用ECD过程开发评估项目，以记录和指导项目的一致性；(2)协调研究，以审查设计模式和任务模板；(3)认知分析研究，以实证调查项目在多大程度上引发预期的指导方针；(4)三项实证研究，包括(A)在第一年对教师(n=6)和学生(n=180)进行的早期测试；(B)在第二年对教师(n=12)和学生(n=360)进行的试点测试；(C)在第三年(n=30)和学生(n=900)进行的主要研究；以及(5)利用教师焦点小组的反馈和对以前研究的学生表现数据的分析，调查评估项目的形成使用情况。项目成果是：(A)经过研究和实地测试的评估原型，衡量学生对两个物理科学核心思想和两个科学和工程实践的思考；(B)研究中使用的相关数据和程序；(C)评估的形成性使用框架，包括准则、评分规则和评估设计决定的标准。

This is a collaborative proposal among the University of Illinois at Chicago, Michigan State University, and SRI International to develop, test, and analyze sets of technology-supported diagnostic classroom assessments for middle school (grades 6-8) physical science. Assessments are aligned with the performance assessment and evidence-centered design methodologies suggested in the Framework for K-12 Science Education (NRC, 2012). The study focuses on the development of new measures of learning that take into account the interdependence of science content and practice. Two disciplinary core ideas--Matter and its Interactions, and Energy--and two scientific and engineering practices--Constructing Explanations and Designing Solutions, and Developing and Using Models--are used for this purpose. The research questions are: (1) What are the characteristic features of science assessments based upon systematic application of the Evidence-Centered Design (ECD) assessment process?; (2) To what extent can assessment designs incorporate critical core idea, crosscutting concept and science/engineering practice dimensions in ways that both separate and integrate these dimensions as part of the design architecture?; (3) What is the evidence that the multiple dimensions of science learning (e.g., content, practices and crosscutting concepts) are separable and recoverable in the performance of students who respond to these assessments?; (4) How instructionally sensitive are these assessments? (i.e., Do they show differential and appropriate sensitivity to students' opportunity to learn science in ways consistent with the vision contained in the NRC Framework?); (5) What forms of evidence can be provided for the validity of these assessments using a multifaceted validity framework that takes into account both the interpretive and evidentiary components of a validity argument for these new assessments?; (6) What are the characteristics of assessments that best serve the needs of classroom teachers relative to a formative assessment process and in what ways do such assessments and scoring processes need to be designed to support effective teacher implementation?; and (7) What are the unique affordances and opportunities provided by technology in designing and implementing assessments focused on merging content & practices performance expectations? Assessments are iteratively designed and administered in three school districts and a laboratory school in Florida and one school district in Wisconsin using the "Investigating and Questioning our World through Science and Technology" curriculum. The three school districts in Florida have classrooms that are using typical curriculum. The assessments will also be administered and tested with students in these classrooms. To address the research questions, the project conducts five major tasks: (1) development of assessment items using the ECD process to document and guide coherence of items; (2) an alignment study to review design patterns and task templates; (3) a cognitive analysis study to empirically investigate the extent to which the items elicit the intended guidelines; (4) three empirical studies, including (a) an early-stage testing with teachers (n=6) and students (n=180) in Year 1, (b) a pilot testing in Year 2 with teachers (n=12) and students (n=360), and (c) a main study in Year 3 with teachers (n=30) and students (n=900); and (5) a study to investigate the formative use of the assessment items using teacher focus groups' feedback and analysis of student performance data from previous studies. Project outcomes are: (a) research-informed and field-tested assessment prototypes that measure students' thinking around the two physical science core ideas and the two scientific and engineering practices; (b) relevant data and procedures used in the studies; and (c) a framework for the formative use of the assessments, including guidelines, scoring rubrics, and criteria for assessment design decisions.