Collaborative Research: Improving the Effectiveness of Practice Exams in Large Introductory Physics Courses

该项目旨在通过提高学生在STEM入门课程中的成功程度来服务于国家利益。对于许多大专院校的学生来说，物理入门课程是他们在STEM领域进一步学习的障碍。每学期约有4000名学生在伊利诺伊大学物理系学习入门课程。大多数学生是工程专业的，因为物理是工程学院所有课程的必修课。尽管教师使用了基于证据的教学方法，如在线预讲、主动学习讲座、“即时教学”、同伴指导和复习部分的协作解决问题，但近15%的学生没有通过入门课程。这些不及格的学生大多积极参加课堂，付出了巨大的努力，但考试成绩很低。当被调查他们认为对学习材料最有帮助的资源时，学生们将带有视频解决方案的练习考试作为课程中评分最高的部分。尽管大量研究表明，模拟考试可能是一种有效的学习形式，但有效性的程度取决于学生参与考试的方式。事实上，简单地回顾问题和解决方案有助于熟悉要测试的材料，但它通常不支持所需的解决问题的能力。因此，表现较差的学生对自己解决问题的能力过于自信，并认为他们已经准备好在考试中取得好成绩，但他们得到的却是不及格的成绩。这个项目将调查改变学生与练习考试互动的形式，以鼓励更早地使用，并提供更准确的理解评估，是否会导致更有效的学习行为。如果是这样的话，它可能会让学生对自己的准备情况做出更好的自我评估，并提高考试表现。这个项目旨在提高三个领域的知识，这三个领域对于提高作为学习资源的练习考试的有效性至关重要，特别是对表现不佳的学生。首先，它将探索学生在STEM入门课程中评估他们的理解水平的困难，也就是众所周知的邓宁·克罗格效应。尽管这一效应在各种情况下都得到了验证，但研究人员不仅会测量这种效应的大小，还会深入了解提高学生评估理解水平的能力是否会改善他们的表现(即这种相关性是因果关系吗？)。其次，该项目将探索实施对学生如何与模拟考试互动以及它们在帮助学生学习方面的有效性所起的作用。特别是，调查人员将试图衡量在模拟考试中为学生提供更现实的分数对他们自己对材料了解程度的准确性的影响。第三，该项目将探索鼓励分布式练习对学生表现的影响，以及鼓励这种练习的相对简单方法的有效性。除了指导未来的教育研究，从这个项目中获得的知识应该最大限度地提高练习考试的有效性，为最需要的高危学生提供练习考试。仅在伊利诺伊大学，这项研究的结果就应该会改善每年数千名希望成为工程师的学生的学习。然而，结果应该转移到其他机构的物理入门课程和其他学科的入门课程，从而扩大这项工作的潜在影响。这个合作项目涉及伊利诺伊大学香槟分校(应获奖-2021099)和普渡大学(应获奖-2021190)的研究人员。该项目由改善本科生STEM教育(IUSE：EHR)计划资助，该计划支持研究和开发项目，以提高所有学生的STEM教育的有效性。通过参与的学生学习路径，该计划支持有前景的实践和工具的创建、探索和实施。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project aims to serve the national interest by improving students' success in introductory STEM courses. For many college and university students, "gateway" courses, such as introductory physics, present a roadblock to further study in STEM fields. Every semester approximately 4,000 students take introductory courses in the Department of Physics at the University of Illinois. Most of the students are engineering majors, because physics is a required course for all programs in the College of Engineering. Despite instructors' use of evidence-based instructional methods such as online pre-lectures, active-learning lectures, "just-in-time teaching," peer instruction, and collaborative problem-solving in recitation sections, nearly 15 percent of students do not pass the introductory courses. Most of these failing students actively participate in class and exert substantial effort, but nevertheless receive low grades on exams. When surveyed about the resources that they find most helpful in learning the material, students cite practice exams with video solutions as the highest-rated component of the course. Although considerable research shows that practice exams can be an effective form of studying, the degree of effectiveness depends on how the students engage with the exams. Indeed, simply reviewing problems and solutions develops familiarity with the material to be tested, but it does not often support the needed problem-solving ability. Consequently, lower-performing students become overconfident in their problem-solving ability and believe they are prepared to do well on exams, but instead they receive failing grades. This project will investigate whether changing the format of students' interactions with the practice exams, to encourage earlier use and to provide a more accurate assessment of understanding, will result in more effective studying behavior. If so, it may enable students to make a better self-assessment of their readiness and increase performance on exams.This project aims to advance knowledge in three areas that are critical for improving the effectiveness of practice exams as a learning resource, especially for underperforming students. First, it will explore students' difficulty in assessing their level of understanding, also known as the Dunning Kroeger effect, in STEM gateway courses. Although this effect has been verified in a variety of situations, the investigators will not only measure the size of the effect but also develop insights about whether improving students' ability to assess their level of understanding improves their performance (i.e., Is the correlation causal?). Second, the project will explore the role that implementation has on how students interact with practice exams and their effectiveness in helping students learn. In particular, the investigators will attempt to measure the impact that providing students a more realistic score on their practice exam has on the accuracy of their own understanding of how well they know the material. Third, the project will explore the impact that encouraging distributed practice can have on student performance, as well as the effectiveness of a relatively simple approach to encouraging that practice. In addition to guiding future education research, the knowledge gained from this project should maximize the effectiveness of practice exams for the at-risk students who need them most. At the University of Illinois alone, the results of the research should improve learning for thousands of students every year who hope to become engineers. However, the results should transfer to introductory physics courses at other institutions and introductory courses in other disciplines, thus magnifying the potential impact of this work. This collaborative project involves investigators at the University of Illinois at Urbana-Champaign (Award DUE-2021099) and Purdue University (Award DUE-2021190). This project is funded by the Improving Undergraduate STEM Education (IUSE: EHR) program, which supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.