Improving the Generalizability of Findings from Educational Evaluations

该项目寻求开发方法来量化和提高STEM教育评估和研究结果的普适性。教育评估的一个中心问题是如何确定评估STEM教育干预效果的精心设计的实验或准实验的结果是否适用于不同的政策相关人群。这项研究将建立在倾向评分方法和国家协变量数据库的基础上，以创建一种统计方法，使用研究样本来估计治疗效果在推断人群中的分布参数。简而言之，这种方法将使使用一项研究的结果在另一人群或地区做出统计声明成为可能。除了新统计方法的数学推导外，该项目还将开发一个公共数据库，其中包含NCES共同核心数据和与之相连的人口普查数据，作为确定推断总体协变值的资源。该方法将通过分析美国国家科学基金会和美国教育部教育科学研究所(IES)支持的20项实地研究的次要数据进行测试和验证。调查员将通过在四个国家教育会议上为其他研究人员举办培训讲习班，提高外地使用新方法的能力。

This project seeks to develop methods to quantify and improve the generalizability of findings from STEM educational evaluation and research. A central problem in educational evaluation is how to determine whether the results of a well-designed experiment or quasi-experiment evaluating the effect of a STEM educational intervention (e.g., curricula, instructional strategies, professional development, or technologies) will apply to a different policy-relevant population. This research will build on propensity score methods and a database of national covariates to create a statistical approach that uses study samples to estimate parameters of the distribution of treatment effects in an inference population. In short, this method will make it possible to use results from one study to make statistical claims in another population or place. This is something that cannot be done in most cases using current methods.In addition to the mathematical derivation of new statistical methods, the project will develop a public-use database containing NCES Common Core of Data and census data linked to it as a resource for establishing covariate values for inference populations. The method will be tested and validated by analyzing secondary data from 20 field studies supported by NSF and the Institute of Education Sciences (IES) of the US Department of Education. The investigator will increase the field's capacity to use the new methods by conducting training workshops at four national educational conferences for other researchers.