Validities of Standards-Based Science Inquiry Assessment

点击翻译按钮获取中文摘要

The proposed research examines the validities of a sample of item formats and performance tasks used in three large-scale science assessments: NAEP, TIMSS, and New Standards and in a set of curriculum-embedded assessments. Qualitative analyses will be used to clarify the alignment of formats used in these assessments with the National Science Education Standards and also to analyze the cognitive demands of the varying assessment formats. Similarly, quantitative methods will supply empirical comparisons of student performance on a sample of different types of items and performance tasks to provide evidence of their validity as measures of scientific inquiry. A conceptual map for understanding and designing balanced indicator systems of science achievement will be developed. The research will offer recommendations for enhancing large-scale science assessment systems .Key Features of the ResearchThe "Validities of Standards-based Science Inquiry Assessments: A Study of Multi-Method Indicator Systems" addresses a timely and important issue: "Just how complete and accurate a picture of science achievement is portrayed by today's high-visibility reference exams such as the Third International Mathematics and Science Study (TIMMS), the National Assessment of Educational Progress (NAEP), or the New Standards Science Assessment? " Despite the substantial investment in the development of high standards and "minds-on/hands-on science curricula" there is limited empirical evidence that large-scale science assessment programs adequately test what students know and can do. If current reference exams are not testing many of the deeper and critical learning outcomes identified in the National Science Education Standards (NSES), the validity of their scores as evidence in support of policies shaping educational programs is questionable. The study will mount three integrated lines of research to study the validities of science assessments: (1) a study of the alignment of a sample of assessment question and exercise formats in the reference exams with with National Science Education Standards, (2) a study of the cognitive demands placed on learners when solving different types of items/tasks drawn from the assessments, and (3) an empirical study of the validities of student scores on combinations of different types of assessment formats drawn from these tests. To provide advice on the conduct of these three research studies, SRI International has assembled a team of nationally recognized experts in the fields of science education, cognitive psychology, and educational measurement. The cognitive analysis, alignment, and empirical strands of the proposed research aim to integrate evidence about intended task demands, actual task elicitations, and confirmatory performance. Two critical outcomes of the project will be: a conceptual framework for characterizing the design features of item/task formats that aim to elicit different types of science learning; and recommendations for the design of multi-level assessment systems that would provide more comprehensive measures of students' science knowledge and inquiry strategies. In sum, this research will help to build a map for understanding and designing balanced indicator systems of science achievement.