Publication Bias in Psychology: A Diagnosis Based on the Correlation between Effect Size and Sample Size
Publication Bias in Psychology: A Diagnosis Based on the Correlation between Effect Size and Sample Size
复制标题
DOI：
10.1371/journal.pone.0105825
复制
发表时间：
2014-09-05
期刊：
影响因子：
3.7
通讯作者：
Scherndl, Thomas
中科院分区：
文献类型：
--
作者：
Kuehberger, Anton;Fritz, Astrid;Scherndl, Thomas
背景：从显著性检验中获得的p值没有提供关于潜在现象的大小或重要性的信息。因此，通常建议额外报告效应大小。效应量在理论上与样本量无关。然而，这在经验上可能并不成立：非独立性可能表明发表偏见。方法：探讨心理学研究中效应量是否独立于样本量。我们从心理学研究的各个领域随机抽取了1000篇心理学文章。我们提取了所有实证论文的p值、效应量和样本量，计算了效应量与样本量的相关性，并考察了p值的分布。结果：我们发现负相关r = - 0.45 [95% CI: - 0.53；- 0.35]效应量和样本量之间的关系。此外，我们还发现刚好超过显著性边界的p值数量异常高。另外的数据表明，隐性和显性权力分析都不能解释这种发现模式。结论：效应量与样本量呈负相关，p值的偏倚分布表明整个心理学领域普遍存在发表偏倚。
Background: The p value obtained from a significance test provides no information about the magnitude or importance of the underlying phenomenon. Therefore, additional reporting of effect size is often recommended. Effect sizes are theoretically independent from sample size. Yet this may not hold true empirically: non-independence could indicate publication bias.Methods: We investigate whether effect size is independent from sample size in psychological research. We randomly sampled 1,000 psychological articles from all areas of psychological research. We extracted p values, effect sizes, and sample sizes of all empirical papers, and calculated the correlation between effect size and sample size, and investigated the distribution of p values.Results: We found a negative correlation of r = -.45 [95% CI: -.53; -.35] between effect size and sample size. In addition, we found an inordinately high number of p values just passing the boundary of significance. Additional data showed that neither implicit nor explicit power analysis could account for this pattern of findings.Conclusion: The negative correlation between effect size and samples size, and the biased distribution of p values indicate pervasive publication bias in the entire field of psychology.