Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say 'Usually Not'
Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say 'Usually Not'
复制标题
经济学研究可复制吗？
DOI：
10.2139/ssrn.2669564
复制
发表时间：
2015
期刊：
影响因子：
--
通讯作者：
Phillip Li
中科院分区：
文献类型：
--
作者：
Andrew C. Chang;Phillip Li
我们试图使用作者提供的复制文件（包括数据和代码）复制发表在13个知名经济学期刊上的67篇论文。我们示例中的一些期刊需要数据和代码复制文件，而其他期刊则不需要这样的文件。除了6篇论文使用机密数据外，35篇论文中有29篇（83%）要求提供数据和代码复制文件作为发表条件，而26篇论文中有11篇（42%）没有要求提供数据和代码复制文件。我们在没有联系作者的情况下成功复制了67篇论文中22篇（33%）的关键定性结果。除去6篇使用保密数据的论文和2篇使用我们没有的软件的论文，我们在作者的帮助下复制了59篇论文中的29篇（49%）。因为即使在作者的帮助下，我们也只能复制样本中不到一半的论文，我们断言经济学研究通常是不可复制的。最后，我们提出了改善经济学研究复制的建议。
We attempt to replicate 67 papers published in 13 well-regarded economics journals using author-provided replication files that include both data and code. Some journals in our sample require data and code replication files, and other journals do not require such files. Aside from 6 papers that use confidential data, we obtain data and code replication files for 29 of 35 papers (83%) that are required to provide such files as a condition of publication, compared to 11 of 26 papers (42%) that are not required to provide data and code replication files. We successfully replicate the key qualitative result of 22 of 67 papers (33%) without contacting the authors. Excluding the 6 papers that use confidential data and the 2 papers that use software we do not possess, we replicate 29 of 59 papers (49%) with assistance from the authors. Because we are able to replicate less than half of the papers in our sample even with help from the authors, we assert that economics research is usually not replicable. We conclude with recommendations on improving replication of economics research.