Validities of Standards-Based Science Inquiry Assessment

拟议的研究考察了三项大规模科学评估（NAEP、TIMSS和新标准）和一套课程嵌入式评估中使用的项目格式和绩效任务样本的有效性。定性分析将用于澄清这些评估中使用的格式与国家科学教育标准的一致性，并分析不同评估格式的认知需求。同样，定量方法将提供学生在不同类型的项目和执行任务的样本上的表现的经验比较，以提供证据证明其作为科学探究措施的有效性。为理解和设计平衡的科学成就指标体系，将开发一个概念图。该研究将为加强大规模科学评估系统提供建议。《基于标准的科学探究评估的有效性：多方法指标体系的研究》提出了一个及时而重要的问题：“今天的高可见度参考考试，如第三次国际数学与科学研究（TIMMS）、国家教育进步评估（NAEP）或新标准科学评估，究竟有多完整和准确地描绘了科学成就的图景？”尽管在高标准和“专注/动手的科学课程”的开发上投入了大量资金，但有限的经验证据表明，大规模的科学评估项目能够充分测试学生的知识和能力。如果目前的参考考试没有测试国家科学教育标准（NSES）中确定的许多更深层次和关键的学习成果，那么他们的分数作为支持制定教育计划的政策的证据的有效性是值得怀疑的。该研究将开展三项综合研究，以研究科学评估的有效性：(1)研究了参考考试中评估问题和练习格式样本与国家科学教育标准的一致性；(2)研究了学习者在解决评估中不同类型的项目/任务时的认知需求；(3)实证研究了学生分数对这些测试中不同类型评估格式组合的有效性。为了对这三项研究的开展提供建议，SRI国际组织了一个由科学教育、认知心理学和教育测量领域的国内知名专家组成的团队。提出的研究的认知分析、一致性和经验链旨在整合有关预期任务需求、实际任务引出和验证性表现的证据。项目的两个关键成果将是：描述旨在引出不同类型科学学习的项目/任务格式设计特征的概念框架；并提出了设计多层次评价体系的建议，为学生的科学知识和探究策略提供更全面的衡量标准。总而言之，本研究有助于构建理解和设计科学成果平衡指标体系的图谱。

The proposed research examines the validities of a sample of item formats and performance tasks used in three large-scale science assessments: NAEP, TIMSS, and New Standards and in a set of curriculum-embedded assessments. Qualitative analyses will be used to clarify the alignment of formats used in these assessments with the National Science Education Standards and also to analyze the cognitive demands of the varying assessment formats. Similarly, quantitative methods will supply empirical comparisons of student performance on a sample of different types of items and performance tasks to provide evidence of their validity as measures of scientific inquiry. A conceptual map for understanding and designing balanced indicator systems of science achievement will be developed. The research will offer recommendations for enhancing large-scale science assessment systems .Key Features of the ResearchThe "Validities of Standards-based Science Inquiry Assessments: A Study of Multi-Method Indicator Systems" addresses a timely and important issue: "Just how complete and accurate a picture of science achievement is portrayed by today's high-visibility reference exams such as the Third International Mathematics and Science Study (TIMMS), the National Assessment of Educational Progress (NAEP), or the New Standards Science Assessment? " Despite the substantial investment in the development of high standards and "minds-on/hands-on science curricula" there is limited empirical evidence that large-scale science assessment programs adequately test what students know and can do. If current reference exams are not testing many of the deeper and critical learning outcomes identified in the National Science Education Standards (NSES), the validity of their scores as evidence in support of policies shaping educational programs is questionable. The study will mount three integrated lines of research to study the validities of science assessments: (1) a study of the alignment of a sample of assessment question and exercise formats in the reference exams with with National Science Education Standards, (2) a study of the cognitive demands placed on learners when solving different types of items/tasks drawn from the assessments, and (3) an empirical study of the validities of student scores on combinations of different types of assessment formats drawn from these tests. To provide advice on the conduct of these three research studies, SRI International has assembled a team of nationally recognized experts in the fields of science education, cognitive psychology, and educational measurement. The cognitive analysis, alignment, and empirical strands of the proposed research aim to integrate evidence about intended task demands, actual task elicitations, and confirmatory performance. Two critical outcomes of the project will be: a conceptual framework for characterizing the design features of item/task formats that aim to elicit different types of science learning; and recommendations for the design of multi-level assessment systems that would provide more comprehensive measures of students' science knowledge and inquiry strategies. In sum, this research will help to build a map for understanding and designing balanced indicator systems of science achievement.