Collaborative Research: Developing Methodological Foundations for Empirical Evaluations of Non-Experimental Methods in STEM Intervention Evaluations

促进评估方法的研究和创新（PRIME）计划旨在支持评估研究，特别强调：（1）探索确定STEM教育项目和计划的影响和有用性的创新方法;（2）建立和扩大评估STEM教育和劳动力发展计划的理论基础，包括翻译和改编其他领域的方法;以及（3）增强评价领域的能力和基础设施。该计划将支持三种类型的提案：探索性项目，包括概念验证和可行性研究;更广泛的全面项目;以及研讨会和会议。拟议的研究仔细注意到上述第1项。 本研究将建立一个连贯的框架，用于设计，实施和分析研究内比较，以评估非实验方法。本研究还将建立一个基础设施，用于对研究内比较设计的结果进行持续的定量合成。本研究将为使用研究内比较设计（WSC）评估STEM评估环境中的非实验方法奠定方法学基础。在WSC设计中，将非实验的治疗效果与共享相同目标人群的随机实验产生的效果进行比较。WSC的目的是确定非实验是否可以复制实验基准的结果，以及这些方法在现场环境中表现良好的背景和条件。该项目有两个总体研究目标。

The Promoting Research and Innovation in Methodologies for Evaluation (PRIME) program seeks to support research on evaluation with special emphasis on: (1) exploring innovative approaches for determining the impacts and usefulness of STEM education projects and programs; (2) building on and expanding the theoretical foundations for evaluating STEM education and workforce development initiatives, including translating and adapting approaches from other fields; and (3) growing the capacity and infrastructure of the evaluation field. Three types of proposals will be supported by the program: Exploratory Projects that include proof-of-concept and feasibility studies; more extensive Full-Scale Projects; and workshops and conferences. The proposed research attends carefully to item 1 above. This research will establish a coherent framework for the design, implementation, and analysis of within-study comparisons for evaluating non-experimental methods. It will also establish an infrastructure for conducting an ongoing quantitative synthesis of results from within-study comparison designs.This research study will develop the methodological foundations for using within-study comparison designs (WSCs) to evaluate non-experimental methods in STEM evaluation settings. In WSC designs, treatment effects from a non-experiment are compared to those produced by a randomized experiment that shares the same target population. The purpose of a WSC is to determine whether the non-experiment can replicate results from an experimental benchmark, and the contexts and conditions under which these methods perform well in field settings. The project has two overarching research aims.