Population validity for educational data mining models: A case study in affect detection
Population validity for educational data mining models: A case study in affect detection
复制标题
教育数据挖掘模型的总体有效性：情感检测的案例研究
DOI：
10.1111/bjet.12156
复制
发表时间：
2014
期刊：
影响因子：
--
通讯作者：
Cristina Heffernan
中科院分区：
文献类型：
--
作者：
Jaclyn L. Ocumpaugh;R. Baker;S. M. Gowda;N. Heffernan;Cristina Heffernan
信息和通信技术（ICT）增强的研究方法，如教育数据挖掘（EDM），使研究人员能够有效地模拟与学生有关的各种结构，从传统的知识评估转向参与评估，元认知，策略和影响。这些结构的自动检测允许EDM研究人员开发可以由软件或教师实施的干预策略。它还允许对结构进行二次分析，其中检测器被应用于比可以通过更传统的方法分析的数据集大得多的数据集。然而，在许多情况下，用于开发EDM模型的数据是从学生那里收集的，他们可能不代表可能使用信通技术的更广泛的人口。为了使用EDM模型（自动检测器）与新的人口，他们的普遍性必须得到验证。在这项研究中，我们检查是否检测器的影响仍然有效时，适用于新的人群。四个教育相关的情感状态的模型构建的基础上，从城市，郊区和农村学生使用ASSISTments软件在美国东北部的中学数学的数据。我们发现，对主要来自一个人口统计学分组的人群进行训练的情感检测器并不能推广到主要来自其他人口统计学分组的人群，即使这些人群可能被认为是同一国家或地区文化的一部分。使用来自所有三个子群体的数据构建的模型比在单个群体上训练的模型更适用于这些群体中的学生，但仍然没有达到理想的群体有效性-在所有子群体中推广的能力。特别是，模型在城市和郊区学生中比农村学生更好地推广。这些发现对数据收集工作、验证技术和旨在大规模应用的干预措施的设计具有重要意义。
Information and communication technology (ICT)-enhanced research methods such as educational data mining (EDM) have allowed researchers to effectively model a broad range of constructs pertaining to the student, moving from traditional assessments of knowledge to assessment of engagement, meta-cognition, strategy and affect. The automated detection of these constructs allows EDM researchers to develop intervention strategies that can be implemented either by the software or the teacher. It also allows for secondary analyses of the construct, where the detectors are applied to a data set that is much larger than one that could be analyzed by more traditional methods. However, in many cases, the data used to develop EDM models are collected from students who may not be representative of the broader populations who are likely to use ICT. In order to use EDM models (automated detectors) with new populations, their generalizability must be verified. In this study, we examine whether detectors of affect remain valid when applied to new populations. Models of four educationally relevant affective states were constructed based on data from urban, suburban and rural students using ASSISTments software for middle school mathematics in the Northeastern United States. We found that affect detectors trained on a population drawn primarily from one demographic grouping do not generalize to populations drawn primarily from the other demographic groupings, even though those populations might be considered part of the same national or regional culture. Models constructed using data from all three subpopulations are more applicable to students in those populations than those trained on a single group, but still do not achieve ideal population validity—the ability to generalize across all subgroups. In particular, models generalize better across urban and suburban students than rural students. These findings have important implications for data collection efforts, validation techniques, and the design of interventions that are intended to be applied at scale.
影响因子：
7.6
作者：
B. Patrick;E. Skinner;J. Connell
通讯作者：
B. Patrick;E. Skinner;J. Connell
DOI：
10.1007/978-3-642-38844-6
发表时间：
2013
期刊：
--
影响因子：
--
作者：
V. Dimitrova;T. Kuflik;David N. Chin;F. Ricci;Peter Dolog;G. Houben
通讯作者：
V. Dimitrova;T. Kuflik;David N. Chin;F. Ricci;Peter Dolog;G. Houben