Obtaining Unbiased Math and Science Achievement Effect Estimates from Nonrandomized Studies

该项目的研究人员将使用大型纵向数据集，其中包括丰富的描述性变量阵列以及科学和数学成就的测量，以确定有希望的协变量，这些协变量可能会消除或大大减少K-12学生研究和干预措施或项目评估中项目效果的非随机估计中的选择偏差。模拟的非随机干预和具有选择偏差的对照组将从五个去识别的、公开可用的纵向数据集中选择数据创建。然后，研究人员将根据协变量与结果之间多重相关性的大小，为每个模拟创建倾向分数，并使用这些分数来评估他们可以消除多少选择偏差。识别足以解释足够选择偏差的协变量，从而可以忽略剩余的选择偏差，这将加强在STEM教育中准实验研究设计的使用。STEM教育中的随机对照设计被认为是最强的因果设计，能够对计划或干预措施对预期结果的影响提供无偏估计。有了足够的样本量，以尽量减少机会差异的可能性，可以期望随机分配在所有可能影响感兴趣的结果的基线特征上使实验组相等。然而，在对STEM教育项目的影响感兴趣的情况下，随机实验并不总是可行的。出于许多实际和偶尔的伦理原因，非随机对照组设计经常用于STEM教育研究和评估研究。本研究的结果将为在准实验设计中应该收集哪些协变量数据以减少选择偏差提供经验证据。

The researchers in this project will use large longitudinal datasets that include rich arrays of descriptive variables as well as measures of science and mathematics achievement to identify promising covariates that may eliminate or greatly reduce selection bias in nonrandomized estimates of program effects in research and evaluations of interventions or programs for K-12 students. Simulated nonrandomized intervention and comparison groups with selection bias will be created from a selection of data from five de-identified, publically available longitudinal datasets. The researchers will then create propensity scores for each simulation based on the examination of the magnitude of the multiple correlations between covariates and the outcome and use these scores to assess how much of the selection bias they might eliminate. The identification of covariates that sufficiently account for enough selection bias so that the remaining selection bias might be ignored will strengthen the use of quasi-experimental study design in STEM education.Randomized control designs in STEM education are well known as being the strongest causal design with the ability to provide unbiased estimates of the effects of programs or interventions on intended outcomes. With an adequate sample size to minimize the likelihood of chance differences, random assignment can be expected to equate the experimental groups on all baseline characteristics that might influence the outcomes of interest. However, randomized experiments are not always possible in situations where the effects of STEM education programs are of interest. For many practical and occasionally, ethical reasons, nonrandomized comparison group designs are frequently used in STEM education research and evaluation studies. The findings of this study will provide empirical evidence of what covariate data should be collected to reduce selection bias in quasi-experimental designs.