Data Analytics for Efficient STEM Assessment: Developing Equivalent Short Concept Inventories

使用经过验证的概念量表是一种公认的评估学生学习的方法，概念量表是一组设计和测试的问题，用于准确地探测学生对特定领域或主题的理解。然而，教师经常避免使用概念清单，因为它们需要大量的课堂时间来完成。本项目旨在精简和修订现有的概念清单，使其缩短版本，以保留其用于学生评估的统计能力。该项目将开发一种标准方法，从目前在STEM教育中使用的既定清单中创建多个缩短的概念清单。生成简短但相关的概念清单将减少使用这些清单所需的课堂时间，也将减轻在课前和课后使用相同的概念清单来衡量学习时可能出现的记忆效应。该项目的结果将提供一种强有力的评估学生学习的新方法。这个为期三年的项目将改进和应用简短的概念清单创建方法，将产生四种常用概念清单的缩短版本-力学基线测试和简短的电和磁评估（均来自物理学），静力学概念清单（来自工程学）和生物学概念清单。统计分析、项目反应理论、因素分析、等同模型和学科专家将指导创建过程，以帮助确保与原始概念清单和缩短版本之间的等同。随机测试将产生数据，以便在新的缩短的概念清单和原来的完整长度测试之间进行比较，以确定多个版本的等效性和在不同版本之间转换分数的适当转换模型。在成功验证后，缩短的概念清单将立即准备好在相关课程中进行管理，并且该方法将适用于所有STEM学科的研究人员的其他概念清单。这项研究还将为实施战略产生基线结果，并指导促进广泛使用缩短的概念清单的努力。在本研究中开发的方法将仔细详细，给出一步一步的指导，以适用于其他研究人员进一步的概念清单。此外，本研究将有助于为未来概念清单的开发提供信息，以使所有教育工作者更容易获得STEM学习评估。

The use of validated concept inventories, which are sets of questions designed and tested to accurately probe student understanding of a particular field or topic, is an accepted method of assessing student learning. However, the use of concept inventories is often avoided by instructors because they take considerable class time to complete. This project seeks to streamline and revise existing concept inventories into shortened versions that retain their statistical power for student assessment. The project will develop a standard method to create multiple shortened concept inventories from established inventories currently used in STEM education. Generating shortened but relevant concept inventories will decrease the class time needed to employ these inventories and will also mitigate memorization effects that can occur when the same concept inventory is used to measure learning before and after a lesson is conducted. Results from this project will provide a powerful new method of assessing student learning.This three-year project to refine and apply short concept inventory creation methodology will result in shortened versions of four commonly used concept inventories - the Mechanics Baseline Test and the Brief Electricity and Magnetism Assessment (both from Physics), the Statics Concept Inventory (from Engineering), and the Biology Concept Inventory. Statistical analysis, item response theory, factor analysis, equating modeling, and discipline-specific experts will guide the creation process to help ensure equivalence to the original concept inventories and between the shortened versions. Randomized testing will produce data that allows for comparisons between new shortened concept inventories and the original full length tests to determine the equivalence of the multiple versions and the appropriate conversion models to translate scores between versions. Upon successful validation, the shortened concept inventories will be immediately ready to administer in relevant courses and the methodology will be applicable to other concept inventories by researchers across all STEM disciplines. This research will also produce baseline outcomes for implementation strategies and guide efforts to promote widespread use of shortened concept inventories. The methodology developed in this research will be carefully detailed, giving step by step instructions to apply to further concept inventories by other researchers. In addition, this research will help to inform future concept inventory development of short parallel concept inventories to make STEM learning assessment more accessible to all educators.