Collaborative Research: Developing Methodological Foundations for Empirical Evaluations of Non-Experimental Methods in STEM Intervention Evaluations

促进评估方法的研究和创新 (PRIME) 计划旨在支持评估研究，特别强调：(1) 探索创新方法来确定 STEM 教育项目和计划的影响和有用性； (2) 建立和扩展评估 STEM 教育和劳动力发展举措的理论基础，包括转化和采用其他领域的方法； (3) 增强评估领域的能力和基础设施。该计划将支持三类提案： 包括概念验证和可行性研究的探索性项目；更广泛的全面项目；以及研讨会和会议。拟议的研究仔细考虑了上述第 1 项。 这项研究将为评估非实验方法的研究内比较的设计、实施和分析建立一个连贯的框架。它还将建立一个基础设施，用于对研究内比较设计的结果进行持续的定量综合。这项研究将为使用研究内比较设计（WSC）来评估 STEM 评估环境中的非实验方法奠定方法基础。在 WSC 设计中，将非实验的治疗效果与具有相同目标人群的随机实验产生的治疗效果进行比较。 WSC 的目的是确定非实验是否可以复制实验基准的结果，以及这些方法在现场设置中表现良好的背景和条件。该项目有两个总体研究目标。

The Promoting Research and Innovation in Methodologies for Evaluation (PRIME) program seeks to support research on evaluation with special emphasis on: (1) exploring innovative approaches for determining the impacts and usefulness of STEM education projects and programs; (2) building on and expanding the theoretical foundations for evaluating STEM education and workforce development initiatives, including translating and adapting approaches from other fields; and (3) growing the capacity and infrastructure of the evaluation field. Three types of proposals will be supported by the program: Exploratory Projects that include proof-of-concept and feasibility studies; more extensive Full-Scale Projects; and workshops and conferences. The proposed research attends carefully to item 1 above. This research will establish a coherent framework for the design, implementation, and analysis of within-study comparisons for evaluating non-experimental methods. It will also establish an infrastructure for conducting an ongoing quantitative synthesis of results from within-study comparison designs.This research study will develop the methodological foundations for using within-study comparison designs (WSCs) to evaluate non-experimental methods in STEM evaluation settings. In WSC designs, treatment effects from a non-experiment are compared to those produced by a randomized experiment that shares the same target population. The purpose of a WSC is to determine whether the non-experiment can replicate results from an experimental benchmark, and the contexts and conditions under which these methods perform well in field settings. The project has two overarching research aims.