Obtaining Unbiased Math and Science Achievement Effect Estimates from Nonrandomized Studies

该项目的研究人员将使用大型纵向数据集，其中包括丰富的描述性变量阵列以及科学和数学成就的衡量标准，以确定有希望的协变量，这些协变量可以消除或极大地减少在研究和评估K-12学生干预或项目时对项目效果的非随机化估计中的选择偏差。模拟的非随机干预和具有选择偏向的对照组将从五个去识别的、可公开获得的纵向数据集中选择数据来创建。然后，研究人员将根据对协变量和结果之间多重相关性的大小的检查，为每个模拟创建倾向分数，并使用这些分数来评估他们可以消除多少选择偏见。确定能够充分解释选择偏差的协变量，以便可以忽略剩余的选择偏差，将加强准实验研究设计在STEM教育中的使用。STEM教育中的随机对照设计被认为是最强的因果设计，能够对项目或干预对预期结果的影响提供无偏见的估计。有了足够的样本量以最大限度地减少机会差异的可能性，可以预期随机分配将使试验组在所有可能影响感兴趣结果的基线特征上相等。然而，在STEM教育项目的效果值得关注的情况下，随机实验并不总是可能的。在STEM教育研究和评估研究中，由于许多实际和偶尔的伦理原因，非随机化比较组设计经常被使用。这项研究的结果将为在准实验设计中收集哪些协变量数据来减少选择偏差提供经验证据。

The researchers in this project will use large longitudinal datasets that include rich arrays of descriptive variables as well as measures of science and mathematics achievement to identify promising covariates that may eliminate or greatly reduce selection bias in nonrandomized estimates of program effects in research and evaluations of interventions or programs for K-12 students. Simulated nonrandomized intervention and comparison groups with selection bias will be created from a selection of data from five de-identified, publically available longitudinal datasets. The researchers will then create propensity scores for each simulation based on the examination of the magnitude of the multiple correlations between covariates and the outcome and use these scores to assess how much of the selection bias they might eliminate. The identification of covariates that sufficiently account for enough selection bias so that the remaining selection bias might be ignored will strengthen the use of quasi-experimental study design in STEM education.Randomized control designs in STEM education are well known as being the strongest causal design with the ability to provide unbiased estimates of the effects of programs or interventions on intended outcomes. With an adequate sample size to minimize the likelihood of chance differences, random assignment can be expected to equate the experimental groups on all baseline characteristics that might influence the outcomes of interest. However, randomized experiments are not always possible in situations where the effects of STEM education programs are of interest. For many practical and occasionally, ethical reasons, nonrandomized comparison group designs are frequently used in STEM education research and evaluation studies. The findings of this study will provide empirical evidence of what covariate data should be collected to reduce selection bias in quasi-experimental designs.