Collaborative Research: Designing Assessments in Physical Science Across Three Dimensions

点击翻译按钮获取中文摘要

This is a collaborative proposal among the University of Illinois at Chicago, Michigan State University, and SRI International to develop, test, and analyze sets of technology-supported diagnostic classroom assessments for middle school (grades 6-8) physical science. Assessments are aligned with the performance assessment and evidence-centered design methodologies suggested in the Framework for K-12 Science Education (NRC, 2012). The study focuses on the development of new measures of learning that take into account the interdependence of science content and practice. Two disciplinary core ideas--Matter and its Interactions, and Energy--and two scientific and engineering practices--Constructing Explanations and Designing Solutions, and Developing and Using Models--are used for this purpose. The research questions are: (1) What are the characteristic features of science assessments based upon systematic application of the Evidence-Centered Design (ECD) assessment process?; (2) To what extent can assessment designs incorporate critical core idea, crosscutting concept and science/engineering practice dimensions in ways that both separate and integrate these dimensions as part of the design architecture?; (3) What is the evidence that the multiple dimensions of science learning (e.g., content, practices and crosscutting concepts) are separable and recoverable in the performance of students who respond to these assessments?; (4) How instructionally sensitive are these assessments? (i.e., Do they show differential and appropriate sensitivity to students' opportunity to learn science in ways consistent with the vision contained in the NRC Framework?); (5) What forms of evidence can be provided for the validity of these assessments using a multifaceted validity framework that takes into account both the interpretive and evidentiary components of a validity argument for these new assessments?; (6) What are the characteristics of assessments that best serve the needs of classroom teachers relative to a formative assessment process and in what ways do such assessments and scoring processes need to be designed to support effective teacher implementation?; and (7) What are the unique affordances and opportunities provided by technology in designing and implementing assessments focused on merging content & practices performance expectations? Assessments are iteratively designed and administered in three school districts and a laboratory school in Florida and one school district in Wisconsin using the "Investigating and Questioning our World through Science and Technology" curriculum. The three school districts in Florida have classrooms that are using typical curriculum. The assessments will also be administered and tested with students in these classrooms. To address the research questions, the project conducts five major tasks: (1) development of assessment items using the ECD process to document and guide coherence of items; (2) an alignment study to review design patterns and task templates; (3) a cognitive analysis study to empirically investigate the extent to which the items elicit the intended guidelines; (4) three empirical studies, including (a) an early-stage testing with teachers (n=6) and students (n=180) in Year 1, (b) a pilot testing in Year 2 with teachers (n=12) and students (n=360), and (c) a main study in Year 3 with teachers (n=30) and students (n=900); and (5) a study to investigate the formative use of the assessment items using teacher focus groups' feedback and analysis of student performance data from previous studies. Project outcomes are: (a) research-informed and field-tested assessment prototypes that measure students' thinking around the two physical science core ideas and the two scientific and engineering practices; (b) relevant data and procedures used in the studies; and (c) a framework for the formative use of the assessments, including guidelines, scoring rubrics, and criteria for assessment design decisions.