Validities of Standards-Based Science Inquiry Assessment

这项拟议的研究考察了三种大规模科学评估：NAEP、TIMSS和新标准以及一套嵌入课程的评估中使用的项目格式和表现任务的样本的有效性。将使用定性分析来澄清这些评估中使用的形式与国家科学教育标准的一致性，并分析不同评估形式的认知需求。同样，定量方法将对学生在不同类型的项目和表现任务样本上的表现进行实证比较，以提供证据证明其作为科学探究衡量标准的有效性。将为理解和设计均衡的科学成就指标体系制定概念图。这项研究将为加强大规模科学评估系统提供建议。这项研究的主要特点：基于标准的科学探究评估的有效性：多方法指标体系的研究“解决了一个及时而重要的问题：今天的高能见度参考考试，如第三次国际数学与科学研究(TIMMS)、国家教育进步评估(NAEP)或新标准科学评估？”尽管在开发高标准和“头脑/动手的科学课程”方面投入了大量资金，但有限的实证证据表明，大规模的科学评估计划足以测试学生的知识和能力。如果目前的参考考试没有测试国家科学教育标准(NSE)中确定的许多更深层次和关键的学习结果，那么作为支持制定教育计划的政策的证据的分数的有效性是值得怀疑的。这项研究将采用三条综合研究路线来研究科学评估的效度：(1)研究参考考试中的评估问题和练习形式样本与国家科学教育标准的一致性；(2)研究从评估中提取不同类型的项目/任务时对学习者的认知需求；以及(3)从这些测试中得出的不同类型评估形式组合上，对学生成绩的有效性进行实证研究。为了就这三项研究的实施提供建议，SRI国际组织了一支由科学教育、认知心理学和教育测量领域的国家公认专家组成的团队。这项研究的认知分析、一致性和经验性分析旨在整合关于预期任务需求、实际任务启发和验证性绩效的证据。该项目的两个关键成果将是：一个概念框架，用于描述旨在引发不同类型科学学习的项目/任务形式的设计特点；以及关于设计多层次评估系统的建议，该系统将为学生的科学知识和探究策略提供更全面的衡量标准。总之，这项研究将有助于构建理解和设计平衡的科学成就指标体系的地图。

The proposed research examines the validities of a sample of item formats and performance tasks used in three large-scale science assessments: NAEP, TIMSS, and New Standards and in a set of curriculum-embedded assessments. Qualitative analyses will be used to clarify the alignment of formats used in these assessments with the National Science Education Standards and also to analyze the cognitive demands of the varying assessment formats. Similarly, quantitative methods will supply empirical comparisons of student performance on a sample of different types of items and performance tasks to provide evidence of their validity as measures of scientific inquiry. A conceptual map for understanding and designing balanced indicator systems of science achievement will be developed. The research will offer recommendations for enhancing large-scale science assessment systems .Key Features of the ResearchThe "Validities of Standards-based Science Inquiry Assessments: A Study of Multi-Method Indicator Systems" addresses a timely and important issue: "Just how complete and accurate a picture of science achievement is portrayed by today's high-visibility reference exams such as the Third International Mathematics and Science Study (TIMMS), the National Assessment of Educational Progress (NAEP), or the New Standards Science Assessment? " Despite the substantial investment in the development of high standards and "minds-on/hands-on science curricula" there is limited empirical evidence that large-scale science assessment programs adequately test what students know and can do. If current reference exams are not testing many of the deeper and critical learning outcomes identified in the National Science Education Standards (NSES), the validity of their scores as evidence in support of policies shaping educational programs is questionable. The study will mount three integrated lines of research to study the validities of science assessments: (1) a study of the alignment of a sample of assessment question and exercise formats in the reference exams with with National Science Education Standards, (2) a study of the cognitive demands placed on learners when solving different types of items/tasks drawn from the assessments, and (3) an empirical study of the validities of student scores on combinations of different types of assessment formats drawn from these tests. To provide advice on the conduct of these three research studies, SRI International has assembled a team of nationally recognized experts in the fields of science education, cognitive psychology, and educational measurement. The cognitive analysis, alignment, and empirical strands of the proposed research aim to integrate evidence about intended task demands, actual task elicitations, and confirmatory performance. Two critical outcomes of the project will be: a conceptual framework for characterizing the design features of item/task formats that aim to elicit different types of science learning; and recommendations for the design of multi-level assessment systems that would provide more comprehensive measures of students' science knowledge and inquiry strategies. In sum, this research will help to build a map for understanding and designing balanced indicator systems of science achievement.