Collaborative Research: Designing Assessments in Physical Science Across Three Dimensions

这是伊利诺伊大学芝加哥分校、密歇根州立大学和SRI国际之间的一项合作提案，旨在开发、测试和分析技术支持的中学（6-8年级）物理科学诊断课堂评估集。评估与K-12科学教育框架（NRC，2012）中建议的绩效评估和以证据为中心的设计方法保持一致。这项研究的重点是制定新的学习措施，考虑到科学内容和实践的相互依存关系。两个学科的核心思想-物质及其相互作用，和能源-和两个科学和工程实践-构建解释和设计解决方案，开发和使用模型-用于此目的。研究问题是：（1）基于循证设计（ECD）评估过程的系统应用的科学评估的特征是什么？(2)评估设计在多大程度上可以将关键核心思想、交叉概念和科学/工程实践维度以既分离又整合这些维度作为设计架构的一部分的方式结合起来？(3)有什么证据表明科学学习的多个维度（例如，内容、实践和交叉概念）在对这些评估作出反应的学生的表现中是可分离和可恢复的？(4)这些评估的敏感性如何？(i.e.,他们是否对学生以符合《核管制委员会框架》所载愿景的方式学习科学的机会表现出不同和适当的敏感性？）; (5)使用一个多方面的有效性框架，考虑到这些新评估的有效性论证的解释和证据成分，可以为这些评估的有效性提供什么形式的证据？(6)相对于形成性评估过程，最能满足课堂教师需求的评估有哪些特点？需要以何种方式设计此类评估和评分过程，以支持教师有效实施？以及（7）在设计和实施侧重于合并内容实践绩效预期的评估时，技术提供了哪些独特的启示和机会？在佛罗里达的三个学区和一所实验学校以及威斯康星州的一个学区，使用“通过科学和技术调查和质疑我们的世界”课程，反复设计和管理评估。佛罗里达的三个学区都有使用典型课程的教室。评估也将在这些教室与学生一起管理和测试。为了解决研究问题，该项目进行了五项主要任务：（1）使用ECD过程开发评估项目，以记录和指导项目的一致性;（2）对齐研究，以审查设计模式和任务模板;（3）认知分析研究，以实证方式调查项目在多大程度上引出预期的指导方针;（4）三项实证研究，包括（a）在一年级对教师（n=6）和学生（n=180）进行的早期测试，（B）在二年级对教师（n=12）和学生（n=360）进行的试点测试，及（c）在三年级进行的一项主要研究，对象为教师（n=30）及学生（n=900）;及（5）一项研究，透过教师焦点小组的反馈及分析以往研究所得的学生表现数据，探讨评估项目的形成性运用。 项目成果是：（a）以研究为基础，并经实地测试的评核原型，以量度学生对两个自然科学核心概念及两个科学和工程实践的思考;（B）研究中使用的相关数据和程序;以及（c）评核的形成性使用框架，包括指引、评分规则，以及评核设计决定的准则。

This is a collaborative proposal among the University of Illinois at Chicago, Michigan State University, and SRI International to develop, test, and analyze sets of technology-supported diagnostic classroom assessments for middle school (grades 6-8) physical science. Assessments are aligned with the performance assessment and evidence-centered design methodologies suggested in the Framework for K-12 Science Education (NRC, 2012). The study focuses on the development of new measures of learning that take into account the interdependence of science content and practice. Two disciplinary core ideas--Matter and its Interactions, and Energy--and two scientific and engineering practices--Constructing Explanations and Designing Solutions, and Developing and Using Models--are used for this purpose. The research questions are: (1) What are the characteristic features of science assessments based upon systematic application of the Evidence-Centered Design (ECD) assessment process?; (2) To what extent can assessment designs incorporate critical core idea, crosscutting concept and science/engineering practice dimensions in ways that both separate and integrate these dimensions as part of the design architecture?; (3) What is the evidence that the multiple dimensions of science learning (e.g., content, practices and crosscutting concepts) are separable and recoverable in the performance of students who respond to these assessments?; (4) How instructionally sensitive are these assessments? (i.e., Do they show differential and appropriate sensitivity to students' opportunity to learn science in ways consistent with the vision contained in the NRC Framework?); (5) What forms of evidence can be provided for the validity of these assessments using a multifaceted validity framework that takes into account both the interpretive and evidentiary components of a validity argument for these new assessments?; (6) What are the characteristics of assessments that best serve the needs of classroom teachers relative to a formative assessment process and in what ways do such assessments and scoring processes need to be designed to support effective teacher implementation?; and (7) What are the unique affordances and opportunities provided by technology in designing and implementing assessments focused on merging content & practices performance expectations? Assessments are iteratively designed and administered in three school districts and a laboratory school in Florida and one school district in Wisconsin using the "Investigating and Questioning our World through Science and Technology" curriculum. The three school districts in Florida have classrooms that are using typical curriculum. The assessments will also be administered and tested with students in these classrooms. To address the research questions, the project conducts five major tasks: (1) development of assessment items using the ECD process to document and guide coherence of items; (2) an alignment study to review design patterns and task templates; (3) a cognitive analysis study to empirically investigate the extent to which the items elicit the intended guidelines; (4) three empirical studies, including (a) an early-stage testing with teachers (n=6) and students (n=180) in Year 1, (b) a pilot testing in Year 2 with teachers (n=12) and students (n=360), and (c) a main study in Year 3 with teachers (n=30) and students (n=900); and (5) a study to investigate the formative use of the assessment items using teacher focus groups' feedback and analysis of student performance data from previous studies. Project outcomes are: (a) research-informed and field-tested assessment prototypes that measure students' thinking around the two physical science core ideas and the two scientific and engineering practices; (b) relevant data and procedures used in the studies; and (c) a framework for the formative use of the assessments, including guidelines, scoring rubrics, and criteria for assessment design decisions.