How Close Is Close Enough? Evaluating Propensity Score Matching Using Data From A Class Size Reduction Experiment
How Close Is Close Enough? Evaluating Propensity Score Matching Using Data From A Class Size Reduction Experiment
复制标题
多接近才算足够接近？
DOI：
--
复制
发表时间：
2007
期刊：
影响因子：
--
通讯作者：
R. Hollister
中科院分区：
文献类型：
--
作者：
E. Wilde;R. Hollister
近年来，倾向得分匹配（PSM）作为一种在缺乏实验评估的情况下估计公共政策项目影响的潜在方法而受到关注。在这项研究中，我们评估了 PSM 在评估教育背景下计划变更的影响方面的有用性（田纳西州的学生教师成就比项目 lProject STARr）。由于田纳西州的 STAR 项目实验涉及有效的随机分配程序，因此该政策干预的实验结果可以用作基准，我们可以将使用倾向得分匹配方法产生的影响估计与该基准进行比较。我们使用几种不同的方法来评估该计划影响的这些非实验估计。我们试图确定“多接近才算足够接近”，最强调这个问题：与程序的实验估计相比，非实验估计会导致错误的决定吗？我们发现倾向评分方法在衡量班级人数减少对成绩测试分数的影响方面表现不佳。我们的结论是，在政策制定者依赖 PSM 作为评估工具之前，需要进一步研究。 © 2007 公共政策分析与管理协会
In recent years, propensity score matching (PSM) has gained attention as a potential method for estimating the impact of public policy programs in the absence of experimental evaluations. In this study, we evaluate the usefulness of PSM for estimating the impact of a program change in an educational context (Tennessee's Student Teacher Achievement Ratio Project lProject STARr). Because Tennessee's Project STAR experiment involved an effective random assignment procedure, the experimental results from this policy intervention can be used as a benchmark, to which we compare the impact estimates produced using propensity score matching methods. We use several different methods to assess these nonexperimental estimates of the impact of the program. We try to determine “how close is close enough,” putting greatest emphasis on the question: Would the nonexperimental estimate have led to the wrong decision when compared to the experimental estimate of the program? We find that propensity score methods perform poorly with respect to measuring the impact of a reduction in class size on achievement test scores. We conclude that further research is needed before policymakers rely on PSM as an evaluation tool. © 2007 by the Association for Public Policy Analysis and Management