Linking Cognitive Science, Measurement Theory and Evaluation Approaches to Assess Development of Scientific Reasoning --- CME

认知，测量和评估（CME）项目将认知科学中关于科学推理及其实现的最新思想与现代仪器开发技术的最新进展相结合。继续医学教育项目将产生一个适合发展的、多项选择和建构反应评估工具的链接系统，旨在衡量各年龄段的科学推理能力。这些工具将是有用的研究人员，教师和教育工作者在设置从五年级到大学的学习者。该项目侧重于控制变量和评估证据的科学推理技能。 这些技能将以适合现代仪器开发的方式加以界定，并将产生一个可供初步实地测试的仪器草案和计算机平台模型。最终的目标是创建一个评估系统，将提供信息：（1）STEM研究人员谁想要了解创新技术，教学方法和/或教学实践如何影响学生的科学推理能力，（2）教师谁需要了解学生是如何响应特定方面的探究教学。参与的机构包括明尼苏达大学和堪萨斯大学及周边学区。基于需求评估，现有的研究和工具以及内容专家的建议，我们将绘制跨越三个年龄组的发展水平的结构图。每个结构图将包括发展水平和每个水平的科学推理的定性描述。项目开发过程将包括评分系统的开发，以映射一定水平的科学推理结构的反应。将通过有声思考、离职面谈和焦点小组面谈对文书初稿进行调查。 使用更大量样本进行的初步试点测试将允许使用IRT对数据进行建模。我们将使用项目拟合、差异项目函数和覆盖图来评估项目属性。我们还将使用项目拟合和模型比较来检查结构的结构。在模型建立阶段，将根据试验数据拟合最简约的Rasch系列模型。如果发现Rasch家族模型不适合数据，将对结构、工具、评分模型和测量模型进行修订。科学教育课程评估最迫切的需求之一是需要适当的测量设备。目前，该领域使用各种评估手段，几乎不可能确定国家影响或比较不同的方法。 虽然科学推理的评估不是一个新概念，但最近关于其定义和测量的想法可能是变革性的。该项目开发的评估工具将得到验证，以确保文化相关性和无偏见，从而加强研究和教育的基础设施。随着时间的推移，使用所开发的工具将使人们能够深入了解不同方案方法对科学推理发展的影响，这是STEM教育的一个关键目标。

The Cognition, Measurement and Evaluation (CME) project combines the latest thinking in cognitive science about scientific reasoning and its attainment with recent advances in modern instrument development techniques. The CME project will produce a linked system of developmentally-appropriate, multiple-choice and constructed-response assessment instruments designed to measure scientific reasoning skills across age levels. These tools will be useful for researchers, classroom teachers and educators in settings for learners from 5th grade through college. The project focuses on scientific reasoning skills of control of variables and evaluating evidence. These skills will be defined in ways suitable for modern instrument development and will result in a draft instrument and model computer platform that would be ready for initial field testing. The ultimate goal is to create an assessment system that will provide information to: (1) STEM researchers who want to understand how innovative technologies, instructional approaches, and/or teaching practices impact students' scientific reasoning abilities, and (2) teachers who need to understand how students are responding to particular aspects of inquiry instruction. The institutions involved include the University of Minnesota and the University of Kansas and surrounding school districts.Based on needs assessment, existing research and instruments and content expert advice, we will map the constructs by developmental level across the three age groups. Each construct map will include developmental levels and a qualitative description of scientific reasoning at each level. The item development process will include development of scoring systems to map responses to certain levels of the scientific reasoning constructs. Initial drafts of the instruments will be investigated through think-alouds, exit interviews, and focus group interviewing. An initial pilot test with a more substantial sample will allow for modeling of the data using IRT. We will use item fit, differential item function, and coverage maps to assess item properties. We will also use item fit and model comparisons to examine the structure of the constructs. During the model-building phase, the Rasch family models, which are the most parsimonious models, will be fitted to the pilot data. If it is discovered that Rasch family models do not fit the data, revisions to the construct, instrument, scoring model, and measurement model will be implemented.One of the most pressing needs in the evaluation of science education programs is the need for appropriate measuring devices. Currently, the field uses a variety of assessment devices, making it almost impossible to determine national effects or to compare different approaches. Although assessment of scientific reasoning is not a new concept, recent ideas about its definition and its measurement may be transformative. The assessment devices developed by this project will be validated to ensure cultural relevancy and absence of bias and therefore will enhance the infrastructure available for research and education. Over time the use of the developed instruments will allow in-depth understanding of the impact of different programmatic approaches to the development of scientific reasoning, a critical goal of STEM education.