Development of Assessment Tools for Evaluating Organic Chemistry Learners' Understanding of Reaction Mechanisms

点击翻译按钮获取中文摘要

This project aims to serve the national interest by developing and implementing computer-based scoring models to evaluate students’ learning about reaction mechanisms in college-level organic chemistry courses. Developing a good understanding of reaction mechanisms is critical for students’ success in organic chemistry courses, which are often a barrier for students seeking to complete STEM degrees and advance their professional goals. To address this challenge, the project team will design predictive scoring models to help instructors provide feedback to students during their studies, as well as to instructors during class sessions. The investigators will collect students’ written explanations for what they think is happening for organic chemistry reaction mechanisms; develop computer-based scoring models using machine learning text analyses to evaluate the written explanations; and offer professional development opportunities (both in-person and asynchronously online) for organic chemistry instructors to reflect upon how they assess students’ understanding of reaction mechanisms and how they can incorporate the new technologies into their organic chemistry courses.The project team from the University of South Florida (USF) will: (1) develop machine learning text analysis scoring models, (2) use the scoring models to explore how student understanding of reaction mechanisms develops across the two-semester organic chemistry course sequence, and (3) facilitate professional development opportunities for organic chemistry instructors. This work will explore how machine learning technologies can be used to develop predictive scoring models for chemistry topics such as the role of nucleophiles or electrophiles in organic chemistry reaction mechanisms. The work will extend current predictive scoring models that only evaluate written explanations for a single assessment prompt to the development of a set of models that can be used to evaluate written explanations for a wide array of prompts. Additionally, this work will capture how faculty members teaching organic chemistry engage with and respond to (1) learners’ written explanations of what is happening in a reaction mechanism and (2) the education research literature that highlights opportunities to improve students’ learning of reaction mechanisms. The project activities have the potential to positively impact the learning and STEM retention of 7,000+ students at the USF study site, an emerging Hispanic-serving institution with a diverse population, including investigating differential impacts on learners for whom English is not their first language. Through this work, the project team will also strive to encourage other researchers to explore more generalized predictive models that address broader concepts and skills (e.g., argumentation and explanation) in STEM courses. The NSF IUSE: EDU program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.