Analysis of an Explainable Student Performance Prediction Model in an Introductory Programming Course
Analysis of an Explainable Student Performance Prediction Model in an Introductory Programming Course
复制标题
DOI：
--
复制
发表时间：
2023
影响因子：
6.7
通讯作者：
Muntasir Hoq;Peter Brusilovsky;Bita Akram
中科院分区：
文献类型：
--
作者：
Muntasir Hoq;Peter Brusilovsky;Bita Akram
在编程入门课程中预测学生的表现可以帮助学习困难的学生，提高他们的毅力。另一方面，预测是透明的，教师和学生有效地利用这种预测的结果是很重要的。可解释的机器学习模型可以有效地帮助学生和教师深入了解学生的不同编程行为和解决问题的策略，这些行为和策略可能会导致学生的表现好坏。本研究建立了一个基于程序作业提交信息的可解释模型来预测学生的表现。我们从学生提交的编程中提取不同的数据驱动特征，并采用堆叠集成模型来预测学生的期末考试成绩。我们使用基于博弈论的SHAP框架来解释模型的预测，以帮助利益相关者了解不同的编程行为对学生成功的影响。此外，我们分析了重要特征的影响，并利用描述性统计和混合模型的组合来识别基于学生问题解决模式的不同概况，以增强可解释性。实验结果表明，我们的模型显著优于其他机器学习模型，包括KNN、SVM、XGBoost、Bagging、Boosting和线性回归。我们的可解释和透明的模型可以帮助解释学生常见的问题解决模式与他们的专业水平之间的关系，从而为学生提供有效的干预和适应性支持。
Prediction of student performance in Introductory programming courses can assist struggling students and improve their persistence. On the other hand, it is important for the prediction to be transparent for the instructor and students to effectively utilize the results of this prediction. Ex-plainable Machine Learning models can effectively help students and instructors gain insights into students’ different programming behaviors and problem-solving strategies that can lead to good or poor performance. This study develops an explainable model that predicts students’ performance based on programming assignment submission information. We extract different data-driven features from students’ programming submissions and employ a stacked ensemble model to predict students’ final exam grades. We use SHAP, a game-theory-based framework, to explain the model’s predictions to help the stakeholders understand the impact of different programming behaviors on students’ success. Moreover, we analyze the impact of important features and utilize a combination of descriptive statistics and mixture models to identify different profiles of students based on their problem-solving patterns to bolster explainability. The experimental results suggest that our model significantly outperforms other Machine Learning models, including KNN, SVM, XGBoost, Bagging, Boosting, and Linear regression. Our explainable and transparent model can help explain students’ common problem-solving patterns in relationship with their level of expertise resulting in effective intervention and adaptive support to students.