Towards investigating the validity of measurement of self-regulated learning based on trace data
Towards investigating the validity of measurement of self-regulated learning based on trace data
复制标题
DOI：
10.1007/s11409-022-09291-1
复制
发表时间：
2022-05-04
影响因子：
3.3
通讯作者：
Gasevic, Dragan
中科院分区：
文献类型：
--
作者：
Fan, Yizhou;van der Graaf, Joep;Gasevic, Dragan
在过去的十年中，将自我调节学习（SRL）视为来自痕迹数据的学习事件的过程的当代研究引起了越来越多的兴趣。然而，已经进行了有限的研究，探讨基于跟踪的测量协议的有效性。为了填补这一空白，在文献中，我们提出了一种新的验证方法，结合理论驱动和数据驱动的观点，以增加解释的有效性SRL过程中提取的跟踪数据。这种方法的主要贡献包括跟踪数据和有声思维数据之间的三个对齐，以提高测量的有效性。此外，我们定义的匹配率之间的SRL过程提取的痕迹数据和大声思考作为一个定量指标与其他三个指标（敏感性，特异性和痕迹覆盖率），以评估的“程度”的有效性。我们在一项实验室研究中测试了这种验证方法，该研究涉及44名学习者，他们使用技术增强的学习环境单独学习人工智能教育主题45分钟。在这种新的验证方法之后，与应用验证方法之前的匹配率（训练集：38.97%;测试集：34.54%）相比，我们实现了从跟踪数据和大声思考数据中提取的SRL过程之间的匹配率（训练集：54.24%;测试集：55.09%）的提高。通过考虑大声思考数据作为“参考点”，这种匹配率的提高量化了使用我们的验证方法可以提高有效性的程度。总之，在这项研究中提出的新的验证方法使用的经验证据，从大声思考的数据和理由，从我们的理论框架的SRL，现在，允许测试和改进的有效性跟踪为基础的SRL测量。
Contemporary research that looks at self-regulated learning (SRL) as processes of learning events derived from trace data has attracted increasing interest over the past decade. However, limited research has been conducted that looks into the validity of trace-based measurement protocols. In order to fill this gap in the literature, we propose a novel validation approach that combines theory-driven and data-driven perspectives to increase the validity of interpretations of SRL processes extracted from trace-data. The main contribution of this approach consists of three alignments between trace data and think aloud data to improve measurement validity. In addition, we define the match rate between SRL processes extracted from trace data and think aloud as a quantitative indicator together with other three indicators (sensitivity, specificity and trace coverage), to evaluate the "degree" of validity. We tested this validation approach in a laboratory study that involved 44 learners who learned individually about the topic of artificial intelligence in education with the use of a technology-enhanced learning environment for 45 minutes. Following this new validation approach, we achieved an improved match rate between SRL processes extracted from trace-data and think aloud data (training set: 54.24%; testing set: 55.09%) compared to the match rate before applying the validation approach (training set: 38.97%; test set: 34.54%). By considering think aloud data as "reference point", this improvement of the match rate quantified the extent to which validity can be improved by using our validation approach. In conclusion, the novel validation approach presented in this study used both empirical evidence from think aloud data and rationale from our theoretical framework of SRL, which now, allows testing and improvement of the validity of trace-based SRL measurements.