Data Analytics for Efficient STEM Assessment: Developing Equivalent Short Concept Inventories

用于高效 STEM 评估的数据分析：开发等效的短概念清单

• 批准号: 1712238
• 负责人: Jing Han
• 金额: $ 29.99万
• 依托单位: Ohio State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-08-01 至 2021-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1712238&HistoricalAwards=false
• 关键词: Data; Analytics; Efficient; STEM; Assessment

The use of validated concept inventories, which are sets of questions designed and tested to accurately probe student understanding of a particular field or topic, is an accepted method of assessing student learning. However, the use of concept inventories is often avoided by instructors because they take considerable class time to complete. This project seeks to streamline and revise existing concept inventories into shortened versions that retain their statistical power for student assessment. The project will develop a standard method to create multiple shortened concept inventories from established inventories currently used in STEM education. Generating shortened but relevant concept inventories will decrease the class time needed to employ these inventories and will also mitigate memorization effects that can occur when the same concept inventory is used to measure learning before and after a lesson is conducted. Results from this project will provide a powerful new method of assessing student learning.This three-year project to refine and apply short concept inventory creation methodology will result in shortened versions of four commonly used concept inventories - the Mechanics Baseline Test and the Brief Electricity and Magnetism Assessment (both from Physics), the Statics Concept Inventory (from Engineering), and the Biology Concept Inventory. Statistical analysis, item response theory, factor analysis, equating modeling, and discipline-specific experts will guide the creation process to help ensure equivalence to the original concept inventories and between the shortened versions. Randomized testing will produce data that allows for comparisons between new shortened concept inventories and the original full length tests to determine the equivalence of the multiple versions and the appropriate conversion models to translate scores between versions. Upon successful validation, the shortened concept inventories will be immediately ready to administer in relevant courses and the methodology will be applicable to other concept inventories by researchers across all STEM disciplines. This research will also produce baseline outcomes for implementation strategies and guide efforts to promote widespread use of shortened concept inventories. The methodology developed in this research will be carefully detailed, giving step by step instructions to apply to further concept inventories by other researchers. In addition, this research will help to inform future concept inventory development of short parallel concept inventories to make STEM learning assessment more accessible to all educators.

使用验证的概念清单是一种公认的评估学生学习情况的方法。验证概念清单是为准确了解学生对特定领域或主题的理解而设计和测试的一组问题。然而，教师通常避免使用概念清单，因为它们需要相当长的课堂时间才能完成。该项目旨在将现有的概念清单精简和修订为简化版本，以保留其对学生评估的统计能力。该项目将开发一种标准方法，从STEM教育中目前使用的已建立的清单中创建多个缩短的概念清单。生成缩短但相关的概念清单将减少使用这些清单所需的课堂时间，还将缓解在授课前后使用相同的概念清单来衡量学习时可能发生的记忆效应。这个项目的结果将为评估学生的学习提供一个强大的新方法。这个为期三年的项目旨在完善和应用简短的概念清单创建方法，将导致四个常用概念清单的缩短版本--力学基线测试和简要的电磁评估(都来自物理学)，静态概念清单(来自工程学)，以及生物概念清单。统计分析、项目反应理论、因素分析、等值建模和特定学科的专家将指导创建过程，以帮助确保与原始概念清单和缩短版本之间的等价性。随机测试将产生数据，以便对新的缩短概念清单和原来的全长测试进行比较，以确定多个版本的等价性和适当的转换模型，以便在不同版本之间转换分数。在成功验证后，缩短的概念清单将立即准备好在相关课程中管理，该方法将适用于所有STEM学科的研究人员的其他概念清单。这项研究还将产生执行战略的基线结果，并指导促进广泛使用缩短概念清单的努力。在这项研究中开发的方法论将被仔细地详述，给出一步一步的指导，以应用于其他研究人员的进一步的概念清单。此外，本研究将有助于为未来概念清单的发展提供信息，使STEM学习评估更容易为所有教育工作者所接受。