Collaborative Research: Improving the Effectiveness of Practice Exams in Large Introductory Physics Courses

协作研究：提高大型物理入门课程练习考试的有效性

• 批准号: 2021190
• 负责人: Jason Morphew
• 金额: $ 3.83万
• 依托单位: Purdue University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2021190&HistoricalAwards=false
• 关键词: Collaborative; Research; Improving; Effectiveness; Practice

This project aims to serve the national interest by improving students' success in introductory STEM courses. For many college and university students, "gateway" courses, such as introductory physics, present a roadblock to further study in STEM fields. Every semester approximately 4,000 students take introductory courses in the Department of Physics at the University of Illinois. Most of the students are engineering majors, because physics is a required course for all programs in the College of Engineering. Despite instructors' use of evidence-based instructional methods such as online pre-lectures, active-learning lectures, "just-in-time teaching," peer instruction, and collaborative problem-solving in recitation sections, nearly 15 percent of students do not pass the introductory courses. Most of these failing students actively participate in class and exert substantial effort, but nevertheless receive low grades on exams. When surveyed about the resources that they find most helpful in learning the material, students cite practice exams with video solutions as the highest-rated component of the course. Although considerable research shows that practice exams can be an effective form of studying, the degree of effectiveness depends on how the students engage with the exams. Indeed, simply reviewing problems and solutions develops familiarity with the material to be tested, but it does not often support the needed problem-solving ability. Consequently, lower-performing students become overconfident in their problem-solving ability and believe they are prepared to do well on exams, but instead they receive failing grades. This project will investigate whether changing the format of students' interactions with the practice exams, to encourage earlier use and to provide a more accurate assessment of understanding, will result in more effective studying behavior. If so, it may enable students to make a better self-assessment of their readiness and increase performance on exams.This project aims to advance knowledge in three areas that are critical for improving the effectiveness of practice exams as a learning resource, especially for underperforming students. First, it will explore students' difficulty in assessing their level of understanding, also known as the Dunning Kroeger effect, in STEM gateway courses. Although this effect has been verified in a variety of situations, the investigators will not only measure the size of the effect but also develop insights about whether improving students' ability to assess their level of understanding improves their performance (i.e., Is the correlation causal?). Second, the project will explore the role that implementation has on how students interact with practice exams and their effectiveness in helping students learn. In particular, the investigators will attempt to measure the impact that providing students a more realistic score on their practice exam has on the accuracy of their own understanding of how well they know the material. Third, the project will explore the impact that encouraging distributed practice can have on student performance, as well as the effectiveness of a relatively simple approach to encouraging that practice. In addition to guiding future education research, the knowledge gained from this project should maximize the effectiveness of practice exams for the at-risk students who need them most. At the University of Illinois alone, the results of the research should improve learning for thousands of students every year who hope to become engineers. However, the results should transfer to introductory physics courses at other institutions and introductory courses in other disciplines, thus magnifying the potential impact of this work. This collaborative project involves investigators at the University of Illinois at Urbana-Champaign (Award DUE-2021099) and Purdue University (Award DUE-2021190). This project is funded by the Improving Undergraduate STEM Education (IUSE: EHR) program, which supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目旨在通过提高学生在STEM入门课程中的成功来服务于国家利益。对于许多学院和大学生来说，“入门”课程，如入门物理学，是进一步学习STEM领域的障碍。每学期大约有4,000名学生在伊利诺伊大学物理系学习入门课程。大部分学生是工科专业，因为物理是工程学院所有课程的必修课。尽管教师使用基于证据的教学方法，如在线预讲座，主动学习讲座，“即时教学”，同伴指导和复习部分的合作解决问题，近15%的学生没有通过入门课程。这些不及格的学生中的大多数积极参与课堂，并付出了巨大的努力，但仍然在考试中获得低分。当被调查他们认为对学习材料最有帮助的资源时，学生们认为带有视频解决方案的练习考试是课程中评分最高的部分。虽然大量的研究表明，模拟考试可以是一种有效的学习形式，但有效性的程度取决于学生如何参与考试。事实上，简单地回顾问题和解决方案可以培养对待测材料的熟悉度，但这通常不支持所需的解决问题的能力。因此，表现较差的学生对自己解决问题的能力过于自信，认为自己已经做好了在考试中取得好成绩的准备，但他们却得到了不及格的成绩。本项目将调查改变学生与练习考试的互动形式，以鼓励更早使用并提供更准确的理解评估，是否会导致更有效的学习行为。 如果是这样的话，它可能使学生能够更好地自我评估他们的准备和提高考试成绩。该项目旨在提高三个领域的知识，这三个领域对于提高练习考试作为学习资源的有效性至关重要，特别是对于表现不佳的学生。首先，它将探讨学生在评估他们的理解水平，也被称为邓宁克罗格效应，在STEM网关课程的困难。虽然这种效果已经在各种情况下得到了验证，但研究人员不仅将测量效果的大小，而且还将深入了解提高学生评估其理解水平的能力是否会提高他们的表现（即，是否存在因果关系？）。其次，该项目将探讨实施对学生如何与练习考试互动的作用，以及它们在帮助学生学习方面的有效性。特别是，研究人员将试图衡量为学生提供更真实的模拟考试分数对他们自己理解材料的准确性的影响。第三，该项目将探讨鼓励分布式实践对学生表现的影响，以及鼓励这种实践的相对简单方法的有效性。除了指导未来的教育研究，从这个项目中获得的知识应该最大限度地提高实践考试的有效性，为最需要他们的高危学生。仅在伊利诺伊大学，研究结果就应该改善每年成千上万希望成为工程师的学生的学习。然而，研究结果应转移到其他机构的物理学入门课程和其他学科的入门课程，从而扩大这项工作的潜在影响。 该合作项目涉及伊利诺伊大学厄巴纳-香槟分校（DUE-2021099奖）和普渡大学（DUE-2021190奖）的研究人员。 该项目由改善本科STEM教育（IUSE：EHR）计划资助，该计划支持研究和开发项目，以提高所有学生STEM教育的有效性。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Impact of more realistic and earlier practice exams on student metacognition, study behaviors, and exam performance

更现实和更早的模拟考试对学生元认知、学习行为和考试表现的影响

• DOI: 10.1103/physrevphyseducres.19.010130
• 发表时间: 2023
• 期刊: Physical Review Physics Education Research
• 影响因子: 3.1
• 作者: Zhang, Muxin;Morphew, Jason;Stelzer, Tim
• 通讯作者: Stelzer, Tim

Effect of online practice exams on student performance

在线模拟考试对学生成绩的影响

• DOI: 10.1119/perc.2019.pr.zhang
• 发表时间: 2020
• 期刊: American Association of Physics Teachers
• 作者: Zhang, Muxin;Engel, Andrew;Stelzer, Timothy;Morphew, Jason W.
• 通讯作者: Morphew, Jason W.

Learning is Quick: Evidence for the Impact of Learning Beliefs on Self-Regulated Learning

学习很快：学习信念对自我调节学习影响的证据

• 发表时间: 2023
• 期刊: American Educational Research Association
• 作者: Morphew, Jason;Shafer, Devyn;Zhang, Muxin;Girotti-Hernandez, Daniela;Stelzer, Tim
• 通讯作者: Stelzer, Tim

Case study on engineering design intervention in physics laboratories

物理实验室工程设计干预案例研究

• 发表时间: 2022
• 期刊: 2022 ASEE Annual Conference & Exposition Proceedings
• 作者: Kaufman-Ortiz, K.;Morphew, J. W.;Rebello, N. S.;& Rebello, C. M.
• 通讯作者: & Rebello, C. M.

Integrated STEM: Impact of Engineering Design and Computer Science in STEM Labs

综合 STEM：工程设计和计算机科学对 STEM 实验室的影响

• DOI: 10.5703/1288284317601
• 发表时间: 2023
• 期刊: Proceedings of the Eighth Annual STEM Education Conference
• 作者: Morphew, Jason;Lopez, Ruben;Bralin, Amir;Subramaniam, Ravishankar;Rebello, Sanjay;Rebello, Carina
• 通讯作者: Rebello, Carina