New Methods for Confusion Detection in Course Forums: Student, Teacher, and Machine
New Methods for Confusion Detection in Course Forums: Student, Teacher, and Machine
复制标题
课程论坛中混淆检测的新方法：学生、教师和机器
DOI：
10.1109/tlt.2021.3123266
复制
发表时间：
2021
影响因子：
3.7
通讯作者：
Karger, David
中科院分区：
文献类型：
--
作者：
Geller, Shay A.;Gal, Kobi;Segal, Avi;Sripathi, Kamali;Kim, Hyunsoo G.;Facciotti, Marc T.;Igo, Michele;Hoernle, Nicholas;Karger, David
这篇文章提供了计算和基于规则的方法来检测学生在课程论坛上的评论中表达的困惑。为了获得关于哪些帖子表现出学生困惑的可靠、真实的数据，我们设计了一棵决策树，便于专家手动标记论坛帖子。然而，手动标记耗费了大量的时间和资源，从而限制了使用此过程可以生成的数据量。我们克服这些限制的策略是，通过反映学生情感状态的标签，根据学生的输入生成检测混淆的规则。我们表明，所得规则与专家的基本事实判断密切一致。接下来，我们将这些规则应用于一门大规模生物学课程中学生论坛帖子的数据集，从而自动生成数千个带有标签的“混淆帖子”实例。最后，得到的数据集被用来训练一个机器学习模型，以检测在没有标签的情况下学生的帖子是否表现出混乱。在这项任务中，基于双向编码器表示的预训练语言模型(BERT)能够优于传统的机器学习模型来对帖子中的混乱进行分类。该模型还能够概括和检测同一课程的不同课程中的学生困惑。最终，这类预先培训的语言模型的使用将为教师提供更好的技术，通过利用教师和学生的联合投入来检测和缓解在线论坛中的困惑。
This article provides computational and rule-based approaches for detecting confusion that is expressed in students' comments in couse forums. To obtain reliable, ground truth data about which posts exhibit student confusion, we designed a decision tree that facilitates the manual labeling of forum posts by experts. However, manual labeling is costly in time and resources, which limits the amount of data that can be generated using this process. Our strategy for overcoming these limitations was to generate rules for detecting confusion based on student input via hashtags, which reflect the student's affective states. We show that the resulting rules closely align with the ground truth judgement of experts. We next applied these rules to datasets of students' forum posts in a large-scale biology course, thereby automatically generating thousands of labeled instances of “confused posts.” Finally, the resulting dataset was used to train a machine learning model for detecting whether students' posts exhibit confusion in the absence of hashtags. In this task, the pretrained language model based on bidirectional encoder representation from transformers (BERT) was able to outperform traditional machine learning models for classifying confusion in posts. This model was also able to generalize and detect student confusion across different offerings of the same course. Ultimately, the use of pretrained language models of this type will provide teachers with better technologies for detecting and alleviating confusion in online discussion forums by leveraging the combined input of teachers and students.