Linking Cognitive Science, Measurement Theory and Evaluation Approaches to Assess Development of Scientific Reasoning --- CME

连接认知科学、测量理论和评估方法来评估科学推理的发展 --- CME

• 批准号: 1118433
• 负责人: Frances Lawrenz
• 金额: $ 23.42万
• 依托单位: University of Minnesota-Twin Cities
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-09-15 至 2014-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1118433&HistoricalAwards=false
• 关键词: Linking; Cognitive; Science; Measurement; Theory

The Cognition, Measurement and Evaluation (CME) project combines the latest thinking in cognitive science about scientific reasoning and its attainment with recent advances in modern instrument development techniques. The CME project will produce a linked system of developmentally-appropriate, multiple-choice and constructed-response assessment instruments designed to measure scientific reasoning skills across age levels. These tools will be useful for researchers, classroom teachers and educators in settings for learners from 5th grade through college. The project focuses on scientific reasoning skills of control of variables and evaluating evidence. These skills will be defined in ways suitable for modern instrument development and will result in a draft instrument and model computer platform that would be ready for initial field testing. The ultimate goal is to create an assessment system that will provide information to: (1) STEM researchers who want to understand how innovative technologies, instructional approaches, and/or teaching practices impact students' scientific reasoning abilities, and (2) teachers who need to understand how students are responding to particular aspects of inquiry instruction. The institutions involved include the University of Minnesota and the University of Kansas and surrounding school districts.Based on needs assessment, existing research and instruments and content expert advice, we will map the constructs by developmental level across the three age groups. Each construct map will include developmental levels and a qualitative description of scientific reasoning at each level. The item development process will include development of scoring systems to map responses to certain levels of the scientific reasoning constructs. Initial drafts of the instruments will be investigated through think-alouds, exit interviews, and focus group interviewing. An initial pilot test with a more substantial sample will allow for modeling of the data using IRT. We will use item fit, differential item function, and coverage maps to assess item properties. We will also use item fit and model comparisons to examine the structure of the constructs. During the model-building phase, the Rasch family models, which are the most parsimonious models, will be fitted to the pilot data. If it is discovered that Rasch family models do not fit the data, revisions to the construct, instrument, scoring model, and measurement model will be implemented.One of the most pressing needs in the evaluation of science education programs is the need for appropriate measuring devices. Currently, the field uses a variety of assessment devices, making it almost impossible to determine national effects or to compare different approaches. Although assessment of scientific reasoning is not a new concept, recent ideas about its definition and its measurement may be transformative. The assessment devices developed by this project will be validated to ensure cultural relevancy and absence of bias and therefore will enhance the infrastructure available for research and education. Over time the use of the developed instruments will allow in-depth understanding of the impact of different programmatic approaches to the development of scientific reasoning, a critical goal of STEM education.

认知、测量和评估（CME）项目结合了认知科学关于科学推理的最新思想及其成就与现代仪器开发技术的最新进展。CME项目将产生一个与发展相适应的、多项选择和建构反应的评估工具相关联的系统，旨在衡量不同年龄层次的科学推理技能。这些工具对于从五年级到大学的学习者的研究人员、课堂教师和教育工作者都很有用。该项目侧重于控制变量和评估证据的科学推理技能。这些技能将以适合现代仪器开发的方式进行定义，并将产生一个仪器草案和模型计算机平台，为初步现场测试做好准备。最终目标是创建一个评估系统，该系统将向：(1)想要了解创新技术、教学方法和/或教学实践如何影响学生科学推理能力的STEM研究人员，以及(2)需要了解学生如何响应探究性教学的特定方面的教师提供信息。涉及的机构包括明尼苏达大学和堪萨斯大学以及周围的学区。根据需求评估、现有研究和工具以及内容专家建议，我们将按三个年龄组的发展水平绘制构建图。每个构建图将包括发展水平和每个水平的科学推理的定性描述。项目开发过程将包括开发评分系统，以映射对科学推理结构的某些级别的反应。将通过大声思考、离职面谈和焦点小组面谈对文书的初稿进行调查。一个具有更大样本的初步试点测试将允许使用IRT对数据进行建模。我们将使用项目适合度、差异项目功能和覆盖图来评估项目属性。我们还将使用项目拟合和模型比较来检查构念的结构。在模型建立阶段，最简洁的Rasch家族模型将被拟合到试验数据中。如果发现Rasch族模型与数据不拟合，将对结构、工具、评分模型和测量模型进行修订。在科学教育项目的评估中，最迫切的需求之一是需要合适的测量设备。目前，该领域使用各种评估手段，几乎不可能确定国家影响或比较不同的方法。虽然科学推理的评估不是一个新概念，但最近关于其定义和测量的想法可能是革命性的。该项目开发的评估设备将经过验证，以确保文化相关性和无偏见，因此将加强研究和教育的基础设施。随着时间的推移，开发工具的使用将使学生深入了解不同方案方法对科学推理发展的影响，科学推理是STEM教育的一个关键目标。