Collaborative Research: Common Error Diagnostics and Support in Short-answer Math Questions

合作研究：简答数学问题中的常见错误诊断和支持

• 批准号: 2118725
• 负责人: Neil Heffernan
• 金额: $ 23.93万
• 依托单位: Worcester Polytechnic Institute
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2118725&HistoricalAwards=false
• 关键词: Collaborative; Research; Common; Error; Diagnostics

One important way to help struggling students improve in math is to deliver personalized support that addresses their specific weaknesses. Many math questions have common wrong answers (CWAs) that correspond to specific errors students make during their answering process, caused by misconceptions or a general lack of knowledge on certain math skills. To date, CWA identification and support remains a labor-intensive process at a limited scale because it requires significant effort by teachers and/or domain experts. In this project, the investigators will develop artificial intelligence (AI)-based mechanisms that can automatically identify CWAs from students’ answers to short-answer math questions and diagnose errors. Once these errors are identified, the investigators will enlist the help of teachers to design feedback and support mechanisms in various formats such as textual feedback messages and short videos. In turn, the investigators will integrate these diagnosis and effective support mechanisms into a teacher interface to support them in either classrooms or online learning environments. Overall, this project has the potential to lead to i) better understanding of CWAs in math questions and the underlying errors and ii) effective CWA support mechanisms for each error type. The project will be grounded in ASSISTments, a free web-based learning platform, therefore directly benefiting the 500,000 US students and 20,000 teachers using it and potentially an even larger number of students and teachers through the dissemination of research findings. This project consists of four main research activities. First, the investigators will leverage math expression embedding methods to learn the representations of student errors by clustering CWAs across multiple questions in the latent math expression embedding vector space. These learned representations will enable the automated diagnosis of student errors in real time. Second, the investigators will develop new knowledge tracing algorithms that go beyond typical correctness analysis and analyze the full answer each student submits to each question. These algorithms will enable the automated tracking of students’ progress in correcting their errors. Third, the investigators will crowdsource multiple types of student support from teachers and integrate both student error diagnostics and support mechanisms into the existing ASSISTments teacher interface. This interface will provide feedback to teachers on which students are struggling in real time and recommend a support, which the teacher can either adopt and customize or reject and create their own support instead. Fourth, the investigators will conduct a randomized controlled trial to evaluate the effectiveness of each support mechanism in helping students correct their errors. This experiment will identify which support mechanisms are most effective at helping students correct each error type and improving learning outcomes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

帮助有困难的学生提高数学成绩的一个重要方法是提供个性化的支持，以解决他们的具体弱点。许多数学问题都有常见的错误答案(CWA)，这些错误答案对应于学生在回答过程中因误解或普遍缺乏某些数学技能知识而犯下的特定错误。到目前为止，CWA的识别和支持仍然是一个有限规模的劳动密集型过程，因为它需要教师和/或领域专家的大量努力。在这个项目中，研究人员将开发基于人工智能(AI)的机制，可以从学生的简答数学问题的答案中自动识别CWA并诊断错误。一旦发现这些错误，调查人员将寻求教师的帮助，以文字反馈消息和短视频等各种格式设计反馈和支持机制。反过来，研究人员将把这些诊断和有效的支持机制整合到教师界面中，以便在课堂或在线学习环境中支持他们。总体而言，这个项目有可能导致i)更好地理解数学问题中的CWA和潜在的错误，以及ii)针对每种错误类型的有效CWA支持机制。该项目将以ASSISTments为基础，这是一个免费的网络学习平台，因此直接受益于使用它的500,000名美国学生和20,000名教师，并可能通过传播研究成果使更多的学生和教师受益。该项目由四项主要研究活动组成。首先，研究人员将利用数学表达式嵌入方法，通过在潜在数学表达式嵌入向量空间中对多个问题的CWA进行聚类，来学习学生错误的表征。这些学习的陈述将使学生错误的实时自动诊断成为可能。其次，调查人员将开发新的知识追踪算法，超越典型的正确性分析，并分析每个学生提交的每个问题的完整答案。这些算法将能够自动跟踪学生在纠正错误方面的进展。第三，调查人员将从教师那里众包多种类型的学生支持，并将学生错误诊断和支持机制整合到现有的ASSISTments教师界面中。这个界面将实时向教师提供反馈，并推荐支持，教师可以采用和定制支持，也可以拒绝并创建自己的支持。第四，研究人员将进行随机对照试验，评估每种支持机制在帮助学生纠正错误方面的有效性。这项实验将确定哪些支持机制在帮助学生纠正每一种错误类型和改善学习结果方面最有效。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Exploring Common Trends in Online Educational Experiments

探索在线教育实验的共同趋势

• 发表时间: 2022
• 期刊: Proceedings of the 15th International Educational Data Mining Conference.
• 作者: Prihar, E.

How to Open Science: Debugging Reproducibility within the Educational Data Mining Conference

如何开放科学：在教育数据挖掘会议中调试再现性

• 发表时间: 2023
• 期刊: EDM 2023
• 作者: Haim, Aaron;Gyurcsan, Robert;Baxter, Chris;Shaw, Stacy;Heffernan, Neil
• 通讯作者: Heffernan, Neil

Using Auxiliary Data to Boost Precision in the Analysis of A/B Tests on an Online Educational Platform: New Data and New Results*

使用辅助数据提高在线教育平台 A/B 测试分析的精度：新数据和新结果*

• 发表时间: 2023
• 期刊: EDM 2023
• 作者: Sales, A.C.

How to Open Science: A Principle and Reproducibility Review of the Learning Analytics and Knowledge Conference

如何开放科学：学习分析和知识会议的原理和可重复性回顾

• DOI: 10.1145/3576050.3576071
• 发表时间: 2023
• 期刊: LAK2023: LAK23: 13th International Learning Analytics and Knowledge Conference
• 作者: Haim, Aaron;Shaw, Stacy;Heffernan, Neil
• 通讯作者: Heffernan, Neil

Investigating the Impact of Skill-Related Videos on Online Learning

调查技能相关视频对在线学习的影响

• DOI: 10.1145/3573051.3593376
• 发表时间: 2023
• 期刊: L@S '23: Proceedings of the Tenth ACM Conference on Learning @ Scale
• 作者: Prihar, Ethan;Haim, Aaron;Shen, Tracy;Sales, Adam;Lee, Dongwon;Wu, Xintao;Heffernan, Neil
• 通讯作者: Heffernan, Neil