Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics
Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics
复制标题
DOI：
10.1257/aer.20190687
复制
发表时间：
2020-11-01
影响因子：
10.7
通讯作者：
Heyes, Anthony
中科院分区：
文献类型：
--
作者：
Brodeur, Abel;Cook, Nikolai;Heyes, Anthony
经济学中的可信性革命促进了随机对照试验（RCT）、差异中差异（DID）、工具变量（IV）和回归不连续设计（RDD）的因果识别。通过对发表在25种主要经济学期刊上的21,000多个假设检验应用多种方法，我们发现p-黑客和出版偏见的程度因方法而异。IV（以及在较小程度上DID）特别成问题。我们没有发现任何证据表明：（i）在前5名期刊上发表的论文与其他期刊不同;（ii）期刊的“修订和重新提交”过程减轻了问题;（iii）随着时间的推移，情况正在改善。
The credibility revolution in economics has promoted causal identification using randomized control trials (RCT), difference-in-differences (DID), instrumental variables (IV) and regression discontinuity design (RDD). Applying multiple approaches to over 21,000 hypothesis tests published in 25 leading economics journals, we find that the extent of p-hacking and publication bias varies greatly by method. IV (and to a lesser extent DID) are particularly problematic. We find no evidence that (i) papers published in the Top 5 journals are different to others; (ii) the journal "revise and resubmit" process mitigates the problem; (iii) things are improving through time.