Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods
Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods
复制标题
DOI：
10.1177/2515245919847196
复制
发表时间：
2019-06-01
影响因子：
13.6
通讯作者：
Hilgard, Joseph
中科院分区：
文献类型：
--
作者：
Carter, Evan C.;Schonbrodt, Felix D.;Hilgard, Joseph
在原始研究中，发表偏倚和可疑的研究实践会导致荟萃分析中严重高估的效果。方法学家提出了各种统计方法来纠正这种高估。然而，目前尚不清楚哪种方法最适合心理学中常见的数据。在这里，我们提出了一个全面的模拟研究，在其中我们研究了一些最有前途的元分析方法是如何对心理学研究可能实际产生的数据进行分析的。我们模拟了几个水平的有问题的研究实践，发表偏倚和异质性，并使用从文献中经验得出的研究样本量。我们的研究结果清楚地表明，没有一种荟萃分析方法始终优于所有其他方法。因此，我们建议心理学中的元分析师关注敏感性分析，也就是说，报告各种方法，考虑这些方法失败的条件（如我们的模拟研究所示），然后报告结论如何根据哪些条件最合理而变化。此外，鉴于元分析方法依赖于不可检验的假设，我们强烈建议心理学研究人员继续努力改进原始文献，并进行大规模的、预先注册的复制。我们在https://osf.io/rf3ys上提供详细的结果和模拟代码，并在http://www.shiny apps.org/apps/metaExplorer/上提供交互式图形。
Publication bias and questionable research practices in primary research can lead to badly overestimated effects in meta-analysis. Methodologists have proposed a variety of statistical approaches to correct for such overestimation. However, it is not clear which methods work best for data typically seen in psychology. Here, we present a comprehensive simulation study in which we examined how some of the most promising meta-analytic methods perform on data that might realistically be produced by research in psychology. We simulated several levels of questionable research practices, publication bias, and heterogeneity, and used study sample sizes empirically derived from the literature. Our results clearly indicated that no single meta-analytic method consistently outperformed all the others. Therefore, we recommend that meta-analysts in psychology focus on sensitivity analyses-that is, report on a variety of methods, consider the conditions under which these methods fail (as indicated by simulation studies such as ours), and then report how conclusions might change depending on which conditions are most plausible. Moreover, given the dependence of meta-analytic methods on untestable assumptions, we strongly recommend that researchers in psychology continue their efforts to improve the primary literature and conduct large-scale, preregistered replications. We provide detailed results and simulation code at https://osf.io/rf3ys and interactive figures at http://www.shiny apps.org/apps/metaExplorer/.