Scaling up behavioral science interventions in online education
Scaling up behavioral science interventions in online education
复制标题
DOI：
10.1073/pnas.1921417117
复制
发表时间：
2020-06-30
影响因子：
11.1
通讯作者：
Tingley, Dustin
中科院分区：
文献类型：
--
作者：
Kizilcec, Rene F.;Reich, Justin;Tingley, Dustin
随着对高等教育和继续教育需求的不断增长，在线教育正在迅速发展，但许多在线学生难以实现他们的教育目标。一些行为科学干预措施在提高学生的坚持性和少数课程的完成率方面显示出了希望，但在不同的教育背景下，其有效性的证据是有限的。在这项研究中，我们测试了一套既定的干预措施超过2.5年，有25万学生，来自几乎每个国家，在247个在线课程由哈佛，马萨诸塞州理工学院，和斯坦福大学提供。我们假设这些干预措施会像以前的研究一样产生中等到大的影响，但我们的结果并不支持这一点。相反，使用一个迭代的科学过程，在数据收集的浪潮之间循环地预先登记新的假设，我们确定了个体，背景和时间条件下，干预措施使学生受益。自我调节干预措施提高了学生在最初几周的参与度，但没有提高最终的完成率。价值相关性干预措施提高了发展中国家的完成率，以缩小全球成绩差距，但仅限于存在全球差距的课程。我们发现最先进的机器学习方法可以预测全球差距的发生或学习有效的个性化干预政策的证据很少。在各种在线学习环境中扩展行为科学干预可以将其平均有效性降低一个数量级。然而，迭代的科学调查可以揭示什么对谁有效。
Online education is rapidly expanding in response to rising demand for higher and continuing education, but many online students struggle to achieve their educational goals. Several behavioral science interventions have shown promise in raising student persistence and completion rates in a handful of courses, but evidence of their effectiveness across diverse educational contexts is limited. In this study, we test a set of established interventions over 2.5 y, with one-quarter million students, from nearly every country, across 247 online courses offered by Harvard, the Massachusetts Institute of Technology, and Stanford. We hypothesized that the interventions would produce medium-to-large effects as in prior studies, but this is not supported by our results. Instead, using an iterative scientific process of cyclically preregistering new hypotheses in between waves of data collection, we identified individual, contextual, and temporal conditions under which the interventions benefit students. Self-regulation interventions raised student engagement in the first few weeks but not final completion rates. Value-relevance interventions raised completion rates in developing countries to close the global achievement gap, but only in courses with a global gap. We found minimal evidence that state-of-the-art machine learning methods can forecast the occurrence of a global gap or learn effective individualized intervention policies. Scaling behavioral science interventions across various online learning contexts can reduce their average effectiveness by an order-of-magnitude. However, iterative scientific investigations can uncover what works where for whom.