A strategy for monitoring and evaluating massive open online courses
A strategy for monitoring and evaluating massive open online courses
复制标题
DOI：
10.1016/j.evalprogplan.2016.04.006
复制
发表时间：
2016-08-01
影响因子：
1.6
通讯作者：
Deacon, A.
中科院分区：
文献类型：
--
作者：
Chapman, S. A.;Goodman, S.;Deacon, A.
我们认为，大规模在线公开课(MOOC)倡议的复杂性、创新性和适应性给监测和评估带来了特别的挑战，因为任何评估战略都需要遵循系统方法。本文旨在通过一系列步骤指导实施MOOC的组织，以帮助他们制定一项战略，以监测、改进和判断其倡议的优点。我们描述了如何通过首先定义给定MOOC系统中交互代理的不同层来实施我们的战略。然后，我们针对这些不同的层面调整我们的方法。具体地说，制定了一种双管齐下的方法，我们建议通过业绩监测对个别项目进行评估；评估标准将在一开始就确定，包括覆盖面、参与度、质量和学生成绩。相比之下，总体倡议的成功应在一个更具适应性、更紧急的评价询问框架内加以考虑。我们介绍了我们为MOOC计划开发的调查框架，并展示了如何使用该框架来开发评估问题和评估方法。我们还为项目绩效监测定义了更固定的指标和措施。我们的战略被描述为为南非开普敦大学(UCT)的MOOC倡议的评估提供信息。(C)2016爱思唯尔有限公司。保留所有权利。
We argue that the complex, innovative and adaptive nature of Massive Open Online Course (MOOC) initiatives poses particular challenges to monitoring and evaluation, in that any evaluation strategy will need to follow a systems approach. This article aims to guide organizations implementing MOOCs through a series of steps to assist them in developing a strategy to monitor, improve, and judge the merit of their initiatives. We describe how we operationalise our strategy by first defining the different layers of interacting agents in a given MOOC system. We then tailor our approach to these different layers. Specifically, a two-pronged approach was developed, where we suggest that individual projects be assessed through performance monitoring; assessment criteria for which would be defined at the outset to include coverage, participation, quality and student achievement. In contrast, the success of an overall initiative should be considered within a more adaptive, emergent evaluation inquiry framework. We present the inquiry framework we developed for MOOC initiatives, and show how this framework might be used to develop evaluation questions and an assessment methodology. We also define the more fixed indicators and measures for project performance monitoring. Our strategy is described as it was developed to inform the evaluation of a MOOC initiative at the University of Cape Town (UCT), South Africa. (C) 2016 Elsevier Ltd. All rights reserved.