Constructing Valid, Equitable, and Flexible Kinematics and Dynamics Assessment Scales with Evidence-Centered Design

该项目旨在通过改进物理入门课程的教与学来服务于国家利益。这个合作项目涉及密歇根州立大学(到期奖金-2235518)、俄亥俄州立大学(到期奖金-2235595)和西弗吉尼亚大学(到期奖金-2235681)的研究人员。在过去的25年里，学生在物理入门课程中的学习经常使用力概念清单(FCI)和力与运动概念评估(FMCE)等工具进行评估，这些工具旨在衡量学生对牛顿力学的概念理解。自那以后，研究人员开始认识到，这些优雅的工具具有可以大大改进的关键特征，包括：i)心理测量特征，这些特征可能限制这些工具在研究和教学应用中的有效性；ii)人口统计学偏见；iii)反映有限多样性的不同时代的艺术品和背景。鉴于这些问题，有必要开发新的和改进的评估工具，重点是在物理入门课程中准确和公平地衡量学生的概念理解。该项目将为教师和教育研究人员提供灵活构建研究型评估工具的机会。这些新的工具将使教师不仅可以确定他们的学生对运动学和动力学的一般理解，而且还可以开发这种理解的细粒度图像，并且这种知识将允许教师将资源引导到最需要的地方。为了取代广泛使用的传统的力学评估(FCI和FMCE)，研究人员的目标是开发一套新的评估，由项目和子量表组成，既提供研究信息，又提供教学信息，并且没有心理测量学限制或人口统计学偏见。他们将利用以证据为中心的设计(ECD)框架从物理学界收集广泛的信息，并构建一系列有效、公平和灵活的项目，组织成分量表。这些项目将在六所大学的大约15,000名学生中进行广泛的验证和测试，以确保女性、未被充分代表的少数族裔学生和第一代大学生具有优越的心理测量特性和工具公平性。一个子量表，由比传统工具少得多的项目组成，没有偏见，将与传统工具交叉规范，以便提供概念性牛顿力学的广泛测量并确保广泛采用。验证过程将被记录下来，以便为物理学界确定有价值的额外评估表的制定奠定基础。NSF IUSE：EDU计划支持研究和开发项目，以提高所有学生的STEM教育的有效性。通过其参与的学生学习跟踪，该计划支持有前途的实践和工具的创建、探索和实施。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project aims to serve the national interest by improving teaching and learning in introductory physics courses. This collaborative project involves investigators at Michigan State University (Award DUE-2235518), Ohio State University (Award DUE-2235595), and West Virginia University (Award DUE-2235681). Over the past 25 years, students' learning in introductory physics courses has often been assessed using instruments such as the Force Concept Inventory (FCI) and the Force and Motion Conceptual Evaluation (FMCE), which were designed to measure students' conceptual understanding of Newtonian mechanics. Researchers have since come to understand that theselegacy instruments have key features that can be greatly improved, including i) psychometric characteristics that may limit the usefulness of those instruments for both research and instructional applications; ii) demographic biases; and iii) artwork and contexts from a different era reflecting limited diversity. In view of these issues, there is a need to develop new and improved assessment tools focused on the accurate and fair measurement of students' conceptual understanding in introductory physics courses. This project will provide instructors and education researchers the opportunity to flexibly construct research-based assessment instruments. These new instruments will allow instructors not only to determine their students' general understanding of kinematics and dynamics but also to develop a fine-grained picture of that understanding, and this knowledge will allow the instructors to direct resources where they are most needed.To replace the widely used legacy assessments for mechanics (the FCI and the FMCE), the investigators aim to develop a new set of assessments composed of items and subscales that are both research-informative and instructionally informative and do not have psychometric limitations or demographic biases. They will leverage the Evidence-Centered Design (ECD) framework to gather broad input from the physics community and construct an array of valid, fair, and flexible items organized into subscales. The items will be extensively validated and tested with approximately 15,000 students at six universities to ensure superior psychometric properties and instrumental fairness for women, underrepresented minority students, and first-generation college students. One subscale, consisting of far fewer items than the legacy instruments, and no bias, will be cross-normed with the legacy instruments in order to provide a broad measurement of conceptual Newtonian mechanics and ensure wide adoption. The validation process will be documented so that it can lay the groundwork for the development of additional assessment scales determined to be of value by the physics community. The NSF IUSE: EDU program supports research and development projects to improve the effectiveness of STEM education for all students. Through its Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.