Mathematical Language Processing: Automatic Grading and Feedback for Open Response Mathematical Questions
Mathematical Language Processing: Automatic Grading and Feedback for Open Response Mathematical Questions
复制标题
数学语言处理：开放式数学问题的自动评分和反馈
DOI：
--
复制
发表时间：
2015
期刊：
影响因子：
--
通讯作者：
Richard Baraniuk
中科院分区：
文献类型：
--
作者：
Andrew S. Lan;Divyanshu Vats;Andrew E. Waters;Richard Baraniuk
尽管计算机和通信技术为扩大教育的许多方面提供了有效手段，但家庭作业和测试等评估的提交和评分仍然是薄弱环节。在本文中，我们研究了对 STEM（科学、技术、工程和数学）课程中常见的开放式数学问题进行自动评分的问题。我们的数据驱动数学语言处理 (MLP) 框架利用大量学习者的解决方案数据来评估其解决方案的正确性、分配部分学分，并向每个学习者提供有关任何错误的可能位置的反馈。 MLP 从文本数据自然语言处理的成功中汲取灵感，包含三个主要步骤。首先，我们将开放式数学问题的每个解决方案转换为一系列数字特征。其次，我们对多个解决方案的特征进行聚类，以揭示正确、部分正确和错误解决方案的结构。我们开发了两种不同的聚类方法，一种利用通用聚类算法，另一种基于贝叶斯非参数。第三，我们根据分配的集群和教师为每个集群提供的一个等级，自动对剩余（可能大量）解决方案进行评分。作为奖励，我们可以跟踪多步解决方案的每个步骤的聚类分配，并确定它何时偏离正确解决方案的聚类，这使我们能够向学习者指示错误的可能位置。我们在真实的 MOOC 数据上测试和验证 MLP，以证明它如何能够大幅减少大型教育平台所需的人力。
While computer and communication technologies have provided effective means to scale up many aspects of education, the submission and grading of assessments such as homework assignments and tests remains a weak link. In this paper, we study the problem of automatically grading the kinds of open response mathematical questions that figure prominently in STEM (science, technology, engineering, and mathematics) courses. Our data-driven framework for mathematical language processing (MLP) leverages solution data from a large number of learners to evaluate the correctness of their solutions, assign partial-credit scores, and provide feedback to each learner on the likely locations of any errors. MLP takes inspiration from the success of natural language processing for text data and comprises three main steps. First, we convert each solution to an open response mathematical question into a series of numerical features. Second, we cluster the features from several solutions to uncover the structures of correct, partially correct, and incorrect solutions. We develop two different clustering approaches, one that leverages generic clustering algorithms and one based on Bayesian nonparametrics. Third, we automatically grade the remaining (potentially large number of) solutions based on their assigned cluster and one instructor-provided grade per cluster. As a bonus, we can track the cluster assignment of each step of a multistep solution and determine when it departs from a cluster of correct solutions, which enables us to indicate the likely locations of errors to learners. We test and validate MLP on real-world MOOC data to demonstrate how it can substantially reduce the human effort required in large-scale educational platforms.