Development of Assessment Tools for Evaluating Organic Chemistry Learners' Understanding of Reaction Mechanisms

开发用于评估有机化学学习者对反应机理理解的评估工具

• 批准号: 2315626
• 负责人: Jeffrey Raker
• 金额: $ 29.91万
• 依托单位: University of South Florida
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2315626&HistoricalAwards=false
• 关键词: Development; Assessment; Tools; Evaluating; Organic

This project aims to serve the national interest by developing and implementing computer-based scoring models to evaluate students’ learning about reaction mechanisms in college-level organic chemistry courses. Developing a good understanding of reaction mechanisms is critical for students’ success in organic chemistry courses, which are often a barrier for students seeking to complete STEM degrees and advance their professional goals. To address this challenge, the project team will design predictive scoring models to help instructors provide feedback to students during their studies, as well as to instructors during class sessions. The investigators will collect students’ written explanations for what they think is happening for organic chemistry reaction mechanisms; develop computer-based scoring models using machine learning text analyses to evaluate the written explanations; and offer professional development opportunities (both in-person and asynchronously online) for organic chemistry instructors to reflect upon how they assess students’ understanding of reaction mechanisms and how they can incorporate the new technologies into their organic chemistry courses.The project team from the University of South Florida (USF) will: (1) develop machine learning text analysis scoring models, (2) use the scoring models to explore how student understanding of reaction mechanisms develops across the two-semester organic chemistry course sequence, and (3) facilitate professional development opportunities for organic chemistry instructors. This work will explore how machine learning technologies can be used to develop predictive scoring models for chemistry topics such as the role of nucleophiles or electrophiles in organic chemistry reaction mechanisms. The work will extend current predictive scoring models that only evaluate written explanations for a single assessment prompt to the development of a set of models that can be used to evaluate written explanations for a wide array of prompts. Additionally, this work will capture how faculty members teaching organic chemistry engage with and respond to (1) learners’ written explanations of what is happening in a reaction mechanism and (2) the education research literature that highlights opportunities to improve students’ learning of reaction mechanisms. The project activities have the potential to positively impact the learning and STEM retention of 7,000+ students at the USF study site, an emerging Hispanic-serving institution with a diverse population, including investigating differential impacts on learners for whom English is not their first language. Through this work, the project team will also strive to encourage other researchers to explore more generalized predictive models that address broader concepts and skills (e.g., argumentation and explanation) in STEM courses. The NSF IUSE: EDU program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目旨在通过开发和实施基于计算机的评分模型来评估学生在大学级有机化学课程中对反应机理的学习情况，以服务于国家利益。对反应机理的深入了解是学生在有机化学课程中取得成功的关键，有机化学课程往往是学生完成STEM学位和推进专业目标的障碍。为了应对这一挑战，项目团队将设计预测性评分模型，以帮助教师在学习期间向学生提供反馈，以及在课堂上向教师提供反馈。调查人员将收集学生对他们认为正在发生的有机化学反应机理的书面解释；利用机器学习文本分析开发基于计算机的评分模型来评估书面解释；并为有机化学教师提供专业发展机会(面对面和异步在线)，以反思他们如何评估学生对反应机理的理解，以及他们如何将新技术融入他们的有机化学课程。来自南佛罗里达大学(USF)的项目团队将：(1)开发机器学习文本分析评分模型，(2)使用评分模型来探索学生对反应机理的理解是如何在两学期的有机化学课程序列中发展的，以及(3)为有机化学教师提供专业发展机会。这项工作将探索如何利用机器学习技术来开发化学主题的预测评分模型，例如亲核剂或亲电剂在有机化学反应机理中的作用。这项工作将把目前只评估单一评估提示的书面解释的预测评分模型扩展到开发一套可用于评估各种提示的书面解释的模型。此外，这项工作将捕捉教授有机化学的教师如何参与和回应(1)学生对反应机制中发生的事情的书面解释，以及(2)强调改善学生学习反应机制的机会的教育研究文献。项目活动有可能对USF学习现场7,000多名学生的学习和STEM保持产生积极影响，USF学习站点是一家新兴的拉美裔服务机构，人口多样化，包括调查对英语不是第一语言的学习者的不同影响。通过这项工作，项目组还将努力鼓励其他研究人员探索更通用的预测模型，以解决STEM课程中更广泛的概念和技能(例如，论证和解释)。NSF IUSE：EDU计划支持研究和开发项目，以提高所有学生的STEM教育的有效性。通过参与的学生学习路径，该计划支持有前景的实践和工具的创建、探索和实施。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。