Algebra Error Classification with Large Language Models
Algebra Error Classification with Large Language Models
复制标题
DOI：
10.48550/arxiv.2305.06163
复制
发表时间：
2023-05
期刊：
影响因子：
--
通讯作者：
Hunter McNichols;Mengxue Zhang;Andrew S. Lan
中科院分区：
文献类型：
--
作者：
Hunter McNichols;Mengxue Zhang;Andrew S. Lan
当学生回答开放式数学问题时，自动反馈在大规模提高学习成果方面具有巨大的潜力。自动反馈系统的一个关键部分是错误分类组件，它可以识别学生的错误，并部署适当的预定义反馈。大多数现有的错误分类方法使用基于规则的方法，这具有有限的泛化能力。现有的数据驱动方法避免了这些限制，但特别需要将学生响应中的数学表达式解析为语法树。这个要求本身就是一个限制，因为学生的回答并不总是语法上有效的，也不能转换成树。在这项工作中，我们介绍了一种使用预训练的大型语言模型进行错误分类的灵活方法。我们证明，我们的方法可以优于现有的方法在代数错误分类，并能够分类一组更大的学生的反应。此外，我们分析了常见的分类错误，我们的方法，并讨论了自动错误分类的局限性。
Automated feedback as students answer open-ended math questions has significant potential in improving learning outcomes at large scale. A key part of automated feedback systems is an error classification component, which identifies student errors and enables appropriate, predefined feedback to be deployed. Most existing approaches to error classification use a rule-based method, which has limited capacity to generalize. Existing data-driven methods avoid these limitations but specifically require mathematical expressions in student responses to be parsed into syntax trees. This requirement is itself a limitation, since student responses are not always syntactically valid and cannot be converted into trees. In this work, we introduce a flexible method for error classification using pre-trained large language models. We demonstrate that our method can outperform existing methods in algebra error classification, and is able to classify a larger set of student responses. Additionally, we analyze common classification errors made by our method and discuss limitations of automated error classification.