Improving the Conceptual Mastery of Engineering Students in High Enrollment Engineering Courses through Oral Exams

该项目旨在通过提高学生对低级别工程课程的概念理解，为未来的成功奠定基础，从而为国家利益服务。掌握基本概念对学生在工程项目中取得成功和坚持不懈至关重要。为了掌握这种技能并为作为工程师的终身学习做好准备，学生们需要制定有效的学习策略。由于评估决定了学生的学习策略，本项目将在高招生的工程入门课程中实施和评估口试。口试将用来评估学生对基本概念的理解，这是很难用传统的考试和测验来衡量的。将设立一个新的培训项目，为教师和助教管理口试做准备。该项目还将提供指导方针，帮助工程教育工作者在其他课程中实施口试。预期口试将引导学生使用学习策略，加强概念掌握和批判性思维。通过加强学生的口头沟通能力，口试也可以帮助他们为成为工程师做好准备。该项目将通过提供有关本科工程核心课程中口语考试有效性的新知识，增加对口试的研究。该项目的目标是提高本科生对工程的概念掌握和批判性思维能力。批判性思维已被确定为工程专业毕业生的基本技能，而概念掌握是先决条件。研究表明，学生采用的学习策略是基于他们将被评估的方式。例如，如果考试本质上是程序性的，学生将追求程序性知识，而不是深刻的概念理解。因此，评估在衡量学生的概念理解水平和指导学生选择学习策略以实现更好的概念掌握和批判性思维技能方面发挥着重要作用。口语考试的适应性对话性质有望改善对概念掌握和学生学习策略的评估。然而，高招生课程口语考试的信度、效度和可扩展性问题尚未得到充分解决。该项目将创建一个框架，将口试纳入核心本科工程课程，补充现有的评估策略。该项目的关键要素是整个教学团队（教员和助教）的参与，并强调培训，其中包括对偏见和残疾学生等问题的考虑。该项目将使用基于证据的评估方法实施迭代设计策略。口头考试的有效性将通过跟踪学生在一门课程的连续口头考试中以及在其他课程中对概念问题的改进来评估。该培训计划将免费提供给其他有兴趣使用口试进行评估的机构的教师。NSF IUSE: EHR计划支持研究和开发项目，以提高所有学生STEM教育的有效性。通过参与学生学习轨道，该计划支持有前途的实践和工具的创建，探索和实施。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project aims to serve the national interest by improving the conceptual understanding of students in the lower division engineering courses that build the foundation for future success. Mastery of basic concepts is essential for student success and perseverance in engineering programs. To gain this mastery and prepare for life-long learning as engineers, students need to develop effective learning strategies. Since assessments drive students’ learning strategies, this project will implement and evaluate oral exams in high enrollment introductory engineering courses. The oral exams will be used to assess students’ understanding of fundamental concepts, which is difficult to measure using traditional exams and quizzes. A new training program will be created to prepare faculty and teaching assistants to administer oral exams. The project will also provide guidelines to help engineering educators implement oral exams in other courses. It is expected that the oral exams will guide students to use learning strategies that enhance conceptual mastery and critical thinking. By strengthening students’ verbal communication skills, oral exams can also help prepare them for the engineering workforce. This project will add to research on oral exams by contributing new knowledge about the effectiveness of oral exams in undergraduate engineering core courses.The goal of the project is to improve undergraduate students’ conceptual mastery and critical thinking skills in engineering. Critical thinking has been identified as an essential skill for engineering graduates and conceptual mastery is a prerequisite. It has been shown that students adopt learning strategies based on how they will be assessed. For example, if an exam is procedural in nature, students will pursue procedural knowledge over deep conceptual understanding. Thus, assessment plays an important role in measuring the level of students' conceptual understanding and in guiding students' selection of learning strategies toward achieving greater concept mastery and critical thinking skills. The adaptive dialogic nature of oral exams holds promise for improving both assessment of conceptual mastery and students’ learning strategies. However, the issues of the reliability, validity, and scalability to high enrollment courses of oral exams have not been fully addressed. This project will create a framework to integrate oral exams in core undergraduate engineering courses, complementing existing assessment strategies. Key elements of the project are the involvement of the entire instructional team (instructors and teaching assistants) and the emphasis on training that includes considerations of issues such as bias and students with disabilities. The project will implement an iterative design strategy using an evidence-based approach of evaluation. The effectiveness of the oral exams will be evaluated by tracking student improvements on conceptual questions across consecutive oral exams in a single course, as well as across other courses. The training program will be made freely available online to faculty at other institutions who are interested in using oral exams for assessment. The NSF IUSE: EHR Program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.