Constructing Valid, Equitable, and Flexible Kinematics and Dynamics Assessment Scales with Evidence-Centered Design

通过以证据为中心的设计构建有效、公平、灵活的运动学和动力学评估量表

• 批准号: 2235681
• 负责人: John Stewart
• 金额: $ 15.1万
• 依托单位: West Virginia University Research Corporation
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2235681&HistoricalAwards=false
• 关键词: Constructing; Valid; Equitable; Flexible; Kinematics

This project aims to serve the national interest by improving teaching and learning in introductory physics courses. This collaborative project involves investigators at Michigan State University (Award DUE-2235518), Ohio State University (Award DUE-2235595), and West Virginia University (Award DUE-2235681). Over the past 25 years, students' learning in introductory physics courses has often been assessed using instruments such as the Force Concept Inventory (FCI) and the Force and Motion Conceptual Evaluation (FMCE), which were designed to measure students' conceptual understanding of Newtonian mechanics. Researchers have since come to understand that theselegacy instruments have key features that can be greatly improved, including i) psychometric characteristics that may limit the usefulness of those instruments for both research and instructional applications; ii) demographic biases; and iii) artwork and contexts from a different era reflecting limited diversity. In view of these issues, there is a need to develop new and improved assessment tools focused on the accurate and fair measurement of students' conceptual understanding in introductory physics courses. This project will provide instructors and education researchers the opportunity to flexibly construct research-based assessment instruments. These new instruments will allow instructors not only to determine their students' general understanding of kinematics and dynamics but also to develop a fine-grained picture of that understanding, and this knowledge will allow the instructors to direct resources where they are most needed.To replace the widely used legacy assessments for mechanics (the FCI and the FMCE), the investigators aim to develop a new set of assessments composed of items and subscales that are both research-informative and instructionally informative and do not have psychometric limitations or demographic biases. They will leverage the Evidence-Centered Design (ECD) framework to gather broad input from the physics community and construct an array of valid, fair, and flexible items organized into subscales. The items will be extensively validated and tested with approximately 15,000 students at six universities to ensure superior psychometric properties and instrumental fairness for women, underrepresented minority students, and first-generation college students. One subscale, consisting of far fewer items than the legacy instruments, and no bias, will be cross-normed with the legacy instruments in order to provide a broad measurement of conceptual Newtonian mechanics and ensure wide adoption. The validation process will be documented so that it can lay the groundwork for the development of additional assessment scales determined to be of value by the physics community. The NSF IUSE: EDU program supports research and development projects to improve the effectiveness of STEM education for all students. Through its Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目旨在通过改善物理入门课程的教学和学习来服务于国家利益。该合作项目涉及密歇根州立大学（Award DUE-2235518）、俄亥俄州州立大学（Award DUE-2235595）和西弗吉尼亚大学（Award DUE-2235681）的研究人员。在过去的25年里，学生在物理入门课程中的学习经常使用诸如力概念清单（FCI）和力与运动概念评估（FMCE）等工具进行评估，这些工具旨在衡量学生对牛顿力学的概念理解。研究人员已经认识到，这些传统工具具有可以大大改进的关键特征，包括i）可能限制这些工具在研究和教学应用中的有用性的心理测量特征; ii）人口统计学偏见; iii）来自不同时代的艺术品和背景，反映了有限的多样性。鉴于这些问题，有必要开发新的和改进的评估工具，侧重于准确和公平的测量学生的概念理解的物理入门课程。该项目将为教师和教育研究人员提供灵活构建基于研究的评估工具的机会。这些新的工具将使教师不仅能够确定学生对运动学和动力学的一般理解，而且还能够对这种理解进行细致的描述，这些知识将使教师能够将资源用于最需要的地方。（FCI和FMCE），研究人员的目标是开发一套新的评估项目和分量表，既研究信息和诊断信息，没有心理测量的限制或人口统计学偏见。他们将利用以证据为中心的设计（ECD）框架，从物理界收集广泛的投入，并构建一系列有效，公平和灵活的项目，组织成子量表。这些项目将在六所大学的大约15 000名学生中进行广泛的验证和测试，以确保女性、代表性不足的少数民族学生和第一代大学生具有上级心理测量特性和工具公平性。一个分量表的项目远少于传统工具，并且没有偏倚，将与传统工具交叉规范，以提供概念牛顿力学的广泛测量并确保广泛采用。将记录验证过程，以便为制定物理学界认为有价值的其他评估量表奠定基础。NSF IUSE：EDU计划支持研究和开发项目，以提高所有学生的STEM教育的有效性。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。