Collaborative Research: Common Error Diagnostics and Support in Short-answer Math Questions

帮助数学困难的学生提高数学成绩的一个重要方法是针对他们的具体弱点提供个性化的支持。许多数学问题都有常见错误答案（CWAs），这些错误与学生在回答过程中犯的特定错误相对应，这些错误是由误解或对某些数学技能的普遍缺乏知识引起的。到目前为止，CWA的识别和支持仍然是一个有限规模的劳动密集型过程，因为它需要教师和/或领域专家的大量努力。在这个项目中，研究人员将开发基于人工智能（AI）的机制，该机制可以自动从学生的简短数学问题的答案中识别CWAs并诊断错误。一旦这些错误被识别出来，研究者将寻求教师的帮助，设计各种形式的反馈和支持机制，如文本反馈信息和短视频。反过来，研究人员将把这些诊断和有效的支持机制整合到教师界面中，以便在课堂或在线学习环境中为他们提供支持。总的来说，这个项目有可能导致i)更好地理解数学问题中的CWA和潜在的错误，ii)每种错误类型的有效CWA支持机制。该项目将以ASSISTments为基础，这是一个免费的基于网络的学习平台，因此可以直接使使用该平台的50万名美国学生和2万名教师受益，并可能通过传播研究成果使更多的学生和教师受益。这个项目包括四个主要的研究活动。首先，研究人员将利用数学表达式嵌入方法，通过在潜在的数学表达式嵌入向量空间中聚类多个问题的cwa来学习学生错误的表示。这些学习到的表征将能够实时自动诊断学生的错误。其次，研究人员将开发新的知识跟踪算法，超越典型的正确性分析，并分析每个学生提交的每个问题的完整答案。这些算法将能够自动跟踪学生在纠正错误方面的进展。第三，研究者将从教师那里获得多种类型的学生支持，并将学生错误诊断和支持机制整合到现有的ASSISTments教师界面中。该界面将实时向教师提供学生遇到困难的反馈，并推荐一种支持，教师可以采用和定制这种支持，也可以拒绝并创建自己的支持。第四，研究者将进行随机对照试验，评估每种支持机制在帮助学生纠正错误方面的有效性。本实验将确定哪种支持机制在帮助学生纠正每种错误类型和提高学习成果方面最有效。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

One important way to help struggling students improve in math is to deliver personalized support that addresses their specific weaknesses. Many math questions have common wrong answers (CWAs) that correspond to specific errors students make during their answering process, caused by misconceptions or a general lack of knowledge on certain math skills. To date, CWA identification and support remains a labor-intensive process at a limited scale because it requires significant effort by teachers and/or domain experts. In this project, the investigators will develop artificial intelligence (AI)-based mechanisms that can automatically identify CWAs from students’ answers to short-answer math questions and diagnose errors. Once these errors are identified, the investigators will enlist the help of teachers to design feedback and support mechanisms in various formats such as textual feedback messages and short videos. In turn, the investigators will integrate these diagnosis and effective support mechanisms into a teacher interface to support them in either classrooms or online learning environments. Overall, this project has the potential to lead to i) better understanding of CWAs in math questions and the underlying errors and ii) effective CWA support mechanisms for each error type. The project will be grounded in ASSISTments, a free web-based learning platform, therefore directly benefiting the 500,000 US students and 20,000 teachers using it and potentially an even larger number of students and teachers through the dissemination of research findings. This project consists of four main research activities. First, the investigators will leverage math expression embedding methods to learn the representations of student errors by clustering CWAs across multiple questions in the latent math expression embedding vector space. These learned representations will enable the automated diagnosis of student errors in real time. Second, the investigators will develop new knowledge tracing algorithms that go beyond typical correctness analysis and analyze the full answer each student submits to each question. These algorithms will enable the automated tracking of students’ progress in correcting their errors. Third, the investigators will crowdsource multiple types of student support from teachers and integrate both student error diagnostics and support mechanisms into the existing ASSISTments teacher interface. This interface will provide feedback to teachers on which students are struggling in real time and recommend a support, which the teacher can either adopt and customize or reject and create their own support instead. Fourth, the investigators will conduct a randomized controlled trial to evaluate the effectiveness of each support mechanism in helping students correct their errors. This experiment will identify which support mechanisms are most effective at helping students correct each error type and improving learning outcomes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.