Contextualized Evaluation Framework for Advanced STEM MOOCs

促进评估方法的研究和创新（PRIME）计划支持评估研究，特别强调：（1）探索确定STEM教育项目和计划的影响和有用性的创新方法;（2）建立和扩大评估STEM教育和劳动力发展计划的理论基础，包括翻译和调整其他领域的方法;以及（3）增强评价领域的能力和基础设施。普渡大学的PI和co-PI将进行一项重要的研究，旨在创建一种评估STEM MOOC的新范式，这种范式超越了分析点击流数据，可用于基于证据的决策。这种新范式的目标是通过考虑学习者特征、课程特征和利益相关者的目标，将评估指标置于情境中。据设想，这种新的范式将指导下一代STEM MOOC的开发。拟议的全规模，为期三年的PRIME项目将开发和测试一个情境化的评估框架，超过56，000名学习者参加了15个高级STEM（纳米技术相关）MOOC，通过普渡X和edX合作伙伴关系，与nanoHUB-U和波音公司合作提供。该项目将导致创建数据收集工具和评估方法，这些工具和方法能够理解各种形式的学习者数据，用于以行动为导向的决策，以支持提供STEM MOOC的学习和机构目标。本研究将采用多阶段混合方法来解决以下研究问题：（1）什么结构有助于学习者？行为和表现在先进的STEM MOOC？(2)需要哪些信息来告知利益相关者关于提供高级STEM MOOC的价值的决策？利益相关者的需求如何推动MOOC的设计？(3)什么是高级STEM MOOC的上下文敏感，可推广的评估框架？ 研究结果有可能通过开发一种新的评估框架，为评估领域做出重大的理论贡献，该框架将更有效地确定提供STEM MOOC的价值和对学习成果的影响。

The Promoting Research and Innovation in Methodologies for Evaluation (PRIME) program supports research on evaluation with special emphasis on: (1) exploring innovative approaches for determining the impacts and usefulness of STEM education projects and programs; (2) building on and expanding the theoretical foundations for evaluating STEM education and workforce development initiatives, including translating and adapting approaches from other fields; and (3) growing the capacity and infrastructure of the evaluation field. The PI and co-PIs at Purdue University will undertake an important research study designed to create a new paradigm for evaluating STEM MOOCs that go beyond analyzing clickstream data and could be used for evidence-based decision making. The goal of this new paradigm is to contextualize evaluation metrics by taking into consideration learner characteristics, course characteristics, and stakeholder goals. It is envisioned that this new paradigm would guide the development of the next generation of STEM MOOCs.The proposed full-scale, three-year PRIME project will develop and test a contextualized evaluation framework on over 56,000 learners enrolled in 15 advanced STEM (nanotechnology-related) MOOCs offered through the Purdue X and edX partnership, in conjunction with nanoHUB-U and Boeing. The project will result in the creation of data collection instruments and an evaluation method that make sense of diverse forms of learner data for action-oriented decisions to support learning and institutional goals in providing STEM MOOCs. The research study will employ a multiphase mixed methods approach to address the following research questions: (1) What constructs contribute to learners? behavior and performance in advanced STEM MOOCs? (2) What information is needed to inform stakeholders decision-making regarding the value of offering advanced STEM MOOCs? How do stakeholders needs drive the design of MOOCs? (3) What is a context sensitive, generalizable framework of evaluation for advanced STEM MOOCs? The results from the research study have the potential to make significant theoretical contributions to the evaluation field by developing a novel evaluation framework that would be more effective for determining the value of offering STEM MOOCs and the impact on learning outcomes.