Constructing Valid, Equitable, and Flexible Kinematics and Dynamics Assessment Scales with Evidence-Centered Design

本项目旨在通过改善物理导论课程的教与学，为国家利益服务。该合作项目涉及密歇根州立大学（奖励DUE-2235518），俄亥俄州立大学（奖励DUE-2235595）和西弗吉尼亚大学（奖励DUE-2235681）的研究人员。在过去的25年里，学生在物理入门课程中的学习情况经常使用诸如力概念量表（FCI）和力与运动概念评估（FMCE）等工具进行评估，这些工具旨在衡量学生对牛顿力学的概念理解。从那以后，研究人员开始认识到，这些传统仪器具有可以大大改进的关键特征，包括i)可能限制这些仪器在研究和教学应用中的有用性的心理测量特征；Ii)人口统计学偏差；iii)来自不同时代的艺术作品和背景，反映出有限的多样性。鉴于这些问题，有必要开发新的和改进的评估工具，重点是准确和公平地衡量学生在物理导论课程中的概念理解。该项目将为教师和教育研究人员提供灵活构建研究型评估工具的机会。这些新工具将使教师不仅能够确定学生对运动学和动力学的总体理解，而且能够对这种理解形成一个精细的图景，这些知识将使教师能够将资源引导到最需要的地方。为了取代广泛使用的力学遗留评估（FCI和FMCE），研究人员旨在开发一套由项目和子量表组成的新的评估，这些评估既具有研究信息又具有教学信息，并且没有心理测量限制或人口统计学偏差。他们将利用以证据为中心的设计（ECD）框架，从物理界收集广泛的意见，并构建一系列有效、公平和灵活的项目，组织成子量表。这些项目将在六所大学的大约15,000名学生中进行广泛的验证和测试，以确保女性、代表性不足的少数民族学生和第一代大学生的优越心理测量特性和工具公平性。一个子量表，包括比传统仪器少得多的项目，没有偏差，将与传统仪器交叉规范，以提供概念牛顿力学的广泛测量，并确保广泛采用。验证过程将被记录下来，以便它可以为物理社区确定的有价值的附加评估量表的开发奠定基础。NSF IUSE: EDU项目支持研究和开发项目，以提高所有学生STEM教育的有效性。通过其参与学生学习轨道，该计划支持有前途的实践和工具的创建，探索和实施。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project aims to serve the national interest by improving teaching and learning in introductory physics courses. This collaborative project involves investigators at Michigan State University (Award DUE-2235518), Ohio State University (Award DUE-2235595), and West Virginia University (Award DUE-2235681). Over the past 25 years, students' learning in introductory physics courses has often been assessed using instruments such as the Force Concept Inventory (FCI) and the Force and Motion Conceptual Evaluation (FMCE), which were designed to measure students' conceptual understanding of Newtonian mechanics. Researchers have since come to understand that theselegacy instruments have key features that can be greatly improved, including i) psychometric characteristics that may limit the usefulness of those instruments for both research and instructional applications; ii) demographic biases; and iii) artwork and contexts from a different era reflecting limited diversity. In view of these issues, there is a need to develop new and improved assessment tools focused on the accurate and fair measurement of students' conceptual understanding in introductory physics courses. This project will provide instructors and education researchers the opportunity to flexibly construct research-based assessment instruments. These new instruments will allow instructors not only to determine their students' general understanding of kinematics and dynamics but also to develop a fine-grained picture of that understanding, and this knowledge will allow the instructors to direct resources where they are most needed.To replace the widely used legacy assessments for mechanics (the FCI and the FMCE), the investigators aim to develop a new set of assessments composed of items and subscales that are both research-informative and instructionally informative and do not have psychometric limitations or demographic biases. They will leverage the Evidence-Centered Design (ECD) framework to gather broad input from the physics community and construct an array of valid, fair, and flexible items organized into subscales. The items will be extensively validated and tested with approximately 15,000 students at six universities to ensure superior psychometric properties and instrumental fairness for women, underrepresented minority students, and first-generation college students. One subscale, consisting of far fewer items than the legacy instruments, and no bias, will be cross-normed with the legacy instruments in order to provide a broad measurement of conceptual Newtonian mechanics and ensure wide adoption. The validation process will be documented so that it can lay the groundwork for the development of additional assessment scales determined to be of value by the physics community. The NSF IUSE: EDU program supports research and development projects to improve the effectiveness of STEM education for all students. Through its Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.