Improving the Generalizability of Findings from Educational Evaluations

该项目旨在开发量化和提高STEM教育评估和研究结果的普遍性的方法。教育评估的一个核心问题是如何确定一个设计良好的实验或准实验的结果，评估STEM教育干预（如课程、教学策略、专业发展或技术）的效果是否适用于不同的政策相关人群。这项研究将建立在倾向评分方法和国家协变量数据库的基础上，以创建一种统计方法，使用研究样本来估计推断人群中治疗效果分布的参数。简而言之，这种方法将有可能利用一项研究的结果在另一个人群或地方做出统计主张。在大多数情况下，使用当前的方法无法做到这一点。除了对新的统计方法进行数学推导外，该项目还将开发一个公共数据库，其中载有NCES共同核心数据和与之相关的普查数据，作为建立推断人口协变量值的资源。该方法将通过分析由美国国家科学基金会和美国教育部教育科学研究所（IES）支持的20项实地研究的二手数据进行测试和验证。该研究员将通过在四个国家教育会议上为其他研究人员举办培训讲习班来提高该领域使用新方法的能力。

This project seeks to develop methods to quantify and improve the generalizability of findings from STEM educational evaluation and research. A central problem in educational evaluation is how to determine whether the results of a well-designed experiment or quasi-experiment evaluating the effect of a STEM educational intervention (e.g., curricula, instructional strategies, professional development, or technologies) will apply to a different policy-relevant population. This research will build on propensity score methods and a database of national covariates to create a statistical approach that uses study samples to estimate parameters of the distribution of treatment effects in an inference population. In short, this method will make it possible to use results from one study to make statistical claims in another population or place. This is something that cannot be done in most cases using current methods.In addition to the mathematical derivation of new statistical methods, the project will develop a public-use database containing NCES Common Core of Data and census data linked to it as a resource for establishing covariate values for inference populations. The method will be tested and validated by analyzing secondary data from 20 field studies supported by NSF and the Institute of Education Sciences (IES) of the US Department of Education. The investigator will increase the field's capacity to use the new methods by conducting training workshops at four national educational conferences for other researchers.