Evaluating the Quality of Learning Resources: A Learnersourcing Approach
Evaluating the Quality of Learning Resources: A Learnersourcing Approach
复制标题
DOI：
10.1109/tlt.2021.3058644
复制
发表时间：
2021-02
影响因子：
3.7
通讯作者：
Solmaz Abdi;Hassan Khosravi;S. Sadiq;Gianluca Demartini
中科院分区：
文献类型：
--
作者：
Solmaz Abdi;Hassan Khosravi;S. Sadiq;Gianluca Demartini
学习者外包正在成为动员学习者社区和利用学习者作为学习资源创造者的智慧的一种可行办法。以前的作品已经表明，由学生开发的资源的质量是相当多样化的一些资源符合严格的判断标准，而其他资源是无效的，不适当的，或不正确的。因此，为了在学生学习中有效地利用这些大型资源库，需要一个选择和调节过程，以将这些资源库中的高质量资源与低质量资源分开。教师和领域专家可能是完成此任务的最可靠来源;然而，他们的可用性通常非常有限。本文探讨了学习者外包作为一种替代方法是否以及如何用于评估学习资源的质量。为此，我们首先采用数据驱动的方法，探索学生判断学习资源质量的能力。这项研究的结果表明，总体而言，学生提供的评级与专家的评级密切相关;然而，学生评估学习资源的能力也可能存在显着差异。然后，我们提出了一种基于矩阵分解的共识方法，并指出它可以用于提高聚合learnersourced决策的准确性。在这篇文章中，我们还演示了如何利用学生的表现，并结合领域专家对有限数量的学习资源的评级可以利用，以进一步提高结果的准确性。
Learnersourcing is emerging as a viable approach for mobilizing the learner community and harnessing the intelligence of learners as creators of learning resources. Previous works have demonstrated that the quality of resources developed by students is quite diverse with some resources meeting rigorous judgmental criteria, whereas other resources are ineffective, inappropriate, or incorrect. Consequently, to effectively utilize these large repositories of resources in student learning, there is a need for a selection and moderation process to separate high-quality resources from low-quality ones in such repositories. Instructors and domain experts are potentially the most reliable source for doing this task; however, their availability is often quite limited. This article explores whether and how learnersourcing, as an alternative approach, can be used for evaluating the quality of learning resources. To do so, we first follow a data-driven approach to explore students’ ability in judging the quality of learning resources. Results from this study suggest that, overall, ratings provided by students strongly correlate with ratings from experts; however, students’ ability in evaluating learning resources can also vary significantly. We then present a consensus approach based on matrix factorization and indicate how it can be used for improving the accuracy of aggregating learnersourced decisions. In this article, we also demonstrate how utilizing information on student performance and incorporating ratings from domain experts on a limited number of learning resources can be leveraged to further improve the accuracy of the results.