Collaborative Research: Improving the Effectiveness of Practice Exams in Large Introductory Physics Courses

该项目旨在通过提高学生在STEM入门课程中的成功程度来服务于国家利益。对于许多大学生来说，“入门”课程，如物理入门，是进一步学习STEM领域的障碍。每学期大约有4000名学生在伊利诺伊大学物理系学习入门课程。大多数学生都是工程专业的，因为物理是工程学院所有专业的必修课。尽管教师使用基于证据的教学方法，如在线预习，主动学习讲座，“即时教学”，同伴指导，以及在背诵部分合作解决问题，近15%的学生没有通过入门课程。这些不及格的学生大多积极参与课堂活动，努力学习，但考试成绩却很低。当被问及他们认为对学习材料最有帮助的资源时，学生们认为带视频解决方案的练习考试是课程中评分最高的部分。尽管相当多的研究表明，练习考试可以是一种有效的学习形式，但有效的程度取决于学生如何参与考试。的确，简单地回顾问题和解决方案可以提高对要测试的材料的熟悉程度，但它通常不支持所需的解决问题的能力。因此，表现较差的学生对自己解决问题的能力变得过于自信，并相信他们准备好了在考试中取得好成绩，但相反，他们得到的是不及格的成绩。该项目将调查改变学生与练习考试的互动形式，以鼓励他们尽早使用，并提供更准确的理解评估，是否会导致更有效的学习行为。如果是这样，它可以使学生更好地自我评估他们的准备情况，提高考试成绩。该项目旨在提高三个领域的知识，这三个领域对于提高实践考试作为学习资源的有效性至关重要，特别是对于表现不佳的学生。首先，它将探索学生在评估自己的理解水平方面的困难，也被称为Dunning Kroeger效应，在STEM入门课程中。虽然这种影响已经在各种情况下得到了验证，但研究人员不仅要测量影响的大小，还要深入了解提高学生评估其理解水平的能力是否会改善他们的表现（即，相关性是否有因果关系？）。其次，该项目将探索实施对学生如何与练习考试互动以及它们在帮助学生学习方面的有效性所起的作用。特别是，调查人员将试图衡量在他们的练习考试中为学生提供一个更现实的分数对他们对自己对材料的了解程度的理解的准确性的影响。第三，该项目将探索鼓励分布式实践对学生表现的影响，以及鼓励这种实践的相对简单方法的有效性。除了指导未来的教育研究之外，从这个项目中获得的知识应该最大限度地为最需要的高危学生提供练习考试的效果。仅在伊利诺伊大学，这项研究的结果每年就应该能改善数千名希望成为工程师的学生的学习。然而，这些结果应该转移到其他机构的物理导论课程和其他学科的导论课程中，从而扩大这项工作的潜在影响。该合作项目涉及伊利诺伊大学厄巴纳-香槟分校（奖励DUE-2021099）和普渡大学（奖励DUE-2021190）的研究人员。该项目由改善本科STEM教育（IUSE: EHR）计划资助，该计划支持研究和开发项目，以提高所有学生STEM教育的有效性。通过参与学生学习轨道，该计划支持有前途的实践和工具的创建，探索和实施。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project aims to serve the national interest by improving students' success in introductory STEM courses. For many college and university students, "gateway" courses, such as introductory physics, present a roadblock to further study in STEM fields. Every semester approximately 4,000 students take introductory courses in the Department of Physics at the University of Illinois. Most of the students are engineering majors, because physics is a required course for all programs in the College of Engineering. Despite instructors' use of evidence-based instructional methods such as online pre-lectures, active-learning lectures, "just-in-time teaching," peer instruction, and collaborative problem-solving in recitation sections, nearly 15 percent of students do not pass the introductory courses. Most of these failing students actively participate in class and exert substantial effort, but nevertheless receive low grades on exams. When surveyed about the resources that they find most helpful in learning the material, students cite practice exams with video solutions as the highest-rated component of the course. Although considerable research shows that practice exams can be an effective form of studying, the degree of effectiveness depends on how the students engage with the exams. Indeed, simply reviewing problems and solutions develops familiarity with the material to be tested, but it does not often support the needed problem-solving ability. Consequently, lower-performing students become overconfident in their problem-solving ability and believe they are prepared to do well on exams, but instead they receive failing grades. This project will investigate whether changing the format of students' interactions with the practice exams, to encourage earlier use and to provide a more accurate assessment of understanding, will result in more effective studying behavior. If so, it may enable students to make a better self-assessment of their readiness and increase performance on exams.This project aims to advance knowledge in three areas that are critical for improving the effectiveness of practice exams as a learning resource, especially for underperforming students. First, it will explore students' difficulty in assessing their level of understanding, also known as the Dunning Kroeger effect, in STEM gateway courses. Although this effect has been verified in a variety of situations, the investigators will not only measure the size of the effect but also develop insights about whether improving students' ability to assess their level of understanding improves their performance (i.e., Is the correlation causal?). Second, the project will explore the role that implementation has on how students interact with practice exams and their effectiveness in helping students learn. In particular, the investigators will attempt to measure the impact that providing students a more realistic score on their practice exam has on the accuracy of their own understanding of how well they know the material. Third, the project will explore the impact that encouraging distributed practice can have on student performance, as well as the effectiveness of a relatively simple approach to encouraging that practice. In addition to guiding future education research, the knowledge gained from this project should maximize the effectiveness of practice exams for the at-risk students who need them most. At the University of Illinois alone, the results of the research should improve learning for thousands of students every year who hope to become engineers. However, the results should transfer to introductory physics courses at other institutions and introductory courses in other disciplines, thus magnifying the potential impact of this work. This collaborative project involves investigators at the University of Illinois at Urbana-Champaign (Award DUE-2021099) and Purdue University (Award DUE-2021190). This project is funded by the Improving Undergraduate STEM Education (IUSE: EHR) program, which supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.