Collaborative Research: Developing Methodological Foundations for Empirical Evaluations of Non-Experimental Methods in STEM Intervention Evaluations

促进评估方法研究和创新(PRIME)计划旨在支持评估研究，特别强调：(1)探索确定STEM教育项目和方案的影响和有用性的创新方法；(2)建立和扩展评估STEM教育和劳动力发展倡议的理论基础，包括翻译和调整其他领域的方法；以及(3)提高评估领域的能力和基础设施。该计划将支持三种类型的提案：包括概念验证和可行性研究的探索性项目；更广泛的全面项目；以及研讨会和会议。拟议的研究仔细考虑了上文第1项。这项研究将为评估非实验方法的研究内比较的设计、实施和分析建立一个连贯的框架。这项研究将为在STEM评估环境中使用研究内比较设计(WSC)来评估非实验方法奠定方法论基础。在WSC设计中，将非实验的治疗效果与共享相同目标人群的随机实验产生的治疗效果进行比较。WSC的目的是确定非实验是否可以复制实验基准的结果，以及这些方法在现场环境中良好运行的背景和条件。该项目有两个主要的研究目标。

The Promoting Research and Innovation in Methodologies for Evaluation (PRIME) program seeks to support research on evaluation with special emphasis on: (1) exploring innovative approaches for determining the impacts and usefulness of STEM education projects and programs; (2) building on and expanding the theoretical foundations for evaluating STEM education and workforce development initiatives, including translating and adapting approaches from other fields; and (3) growing the capacity and infrastructure of the evaluation field. Three types of proposals will be supported by the program: Exploratory Projects that include proof-of-concept and feasibility studies; more extensive Full-Scale Projects; and workshops and conferences. The proposed research attends carefully to item 1 above. This research will establish a coherent framework for the design, implementation, and analysis of within-study comparisons for evaluating non-experimental methods. It will also establish an infrastructure for conducting an ongoing quantitative synthesis of results from within-study comparison designs.This research study will develop the methodological foundations for using within-study comparison designs (WSCs) to evaluate non-experimental methods in STEM evaluation settings. In WSC designs, treatment effects from a non-experiment are compared to those produced by a randomized experiment that shares the same target population. The purpose of a WSC is to determine whether the non-experiment can replicate results from an experimental benchmark, and the contexts and conditions under which these methods perform well in field settings. The project has two overarching research aims.