Collaborative Research: Common Error Diagnostics and Support in Short-answer Math Questions

帮助苦苦挣扎的学生提高数学水平的一个重要方法是提供个性化的支持，以解决他们的具体弱点。许多数学问题都有常见的错误答案（CWA），这与学生在回答过程中犯的特定错误相对应，这些错误是由误解或对某些数学技能的普遍缺乏知识造成的。到目前为止，CWA的识别和支持仍然是一个劳动密集型的过程，在有限的规模，因为它需要教师和/或领域专家的显着努力。在这个项目中，研究人员将开发基于人工智能（AI）的机制，可以从学生对简答数学问题的回答中自动识别CWA并诊断错误。一旦发现这些错误，调查人员将寻求教师的帮助，以文本反馈信息和短视频等各种形式设计反馈和支持机制。反过来，研究人员将这些诊断和有效的支持机制整合到教师界面中，以支持他们在课堂或在线学习环境中。总的来说，这个项目有可能导致i）更好地理解数学问题中的CWA和潜在的错误，ii）针对每种错误类型的有效CWA支持机制。该项目将以ASSISTments为基础，这是一个免费的基于网络的学习平台，因此直接受益于50万美国学生和2万名教师使用它，并可能通过传播研究成果使更多的学生和教师受益。该项目包括四项主要研究活动。首先，研究人员将利用数学表达式嵌入方法，通过在潜在数学表达式嵌入向量空间中跨多个问题聚类CWA来学习学生错误的表示。这些学习到的表示将使自动诊断学生的错误在真实的时间。其次，研究人员将开发新的知识追踪算法，超越典型的正确性分析，并分析每个学生提交的每个问题的完整答案。这些算法将能够自动跟踪学生在纠正错误方面的进展。第三，研究人员将从教师那里众包多种类型的学生支持，并将学生错误诊断和支持机制整合到现有的ASSISTments教师界面中。该界面将向教师提供关于学生在真实的时间中挣扎的反馈，并推荐支持，教师可以采用和定制或拒绝并创建他们自己的支持。第四，研究人员将进行随机对照试验，以评估每种支持机制在帮助学生纠正错误方面的有效性。该实验将确定哪些支持机制在帮助学生纠正每种错误类型和提高学习成果方面最有效。该奖项反映了NSF的法定使命，并且通过使用基金会的知识价值和更广泛的影响进行评估，被认为值得支持审查标准。

One important way to help struggling students improve in math is to deliver personalized support that addresses their specific weaknesses. Many math questions have common wrong answers (CWAs) that correspond to specific errors students make during their answering process, caused by misconceptions or a general lack of knowledge on certain math skills. To date, CWA identification and support remains a labor-intensive process at a limited scale because it requires significant effort by teachers and/or domain experts. In this project, the investigators will develop artificial intelligence (AI)-based mechanisms that can automatically identify CWAs from students’ answers to short-answer math questions and diagnose errors. Once these errors are identified, the investigators will enlist the help of teachers to design feedback and support mechanisms in various formats such as textual feedback messages and short videos. In turn, the investigators will integrate these diagnosis and effective support mechanisms into a teacher interface to support them in either classrooms or online learning environments. Overall, this project has the potential to lead to i) better understanding of CWAs in math questions and the underlying errors and ii) effective CWA support mechanisms for each error type. The project will be grounded in ASSISTments, a free web-based learning platform, therefore directly benefiting the 500,000 US students and 20,000 teachers using it and potentially an even larger number of students and teachers through the dissemination of research findings. This project consists of four main research activities. First, the investigators will leverage math expression embedding methods to learn the representations of student errors by clustering CWAs across multiple questions in the latent math expression embedding vector space. These learned representations will enable the automated diagnosis of student errors in real time. Second, the investigators will develop new knowledge tracing algorithms that go beyond typical correctness analysis and analyze the full answer each student submits to each question. These algorithms will enable the automated tracking of students’ progress in correcting their errors. Third, the investigators will crowdsource multiple types of student support from teachers and integrate both student error diagnostics and support mechanisms into the existing ASSISTments teacher interface. This interface will provide feedback to teachers on which students are struggling in real time and recommend a support, which the teacher can either adopt and customize or reject and create their own support instead. Fourth, the investigators will conduct a randomized controlled trial to evaluate the effectiveness of each support mechanism in helping students correct their errors. This experiment will identify which support mechanisms are most effective at helping students correct each error type and improving learning outcomes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.