An empirical analysis of journal policy effectiveness for computational reproducibility
An empirical analysis of journal policy effectiveness for computational reproducibility
复制标题
DOI：
10.1073/pnas.1708290115
复制
发表时间：
2018-03-13
影响因子：
11.1
通讯作者：
Ma, Zhaokun
中科院分区：
文献类型：
--
作者：
Stodden, Victoria;Seiler, Jennifer;Ma, Zhaokun
科学交流的一个关键组成部分是为该领域的其他研究人员提供足够的信息，以复制已发表的发现。对于计算型和数据型研究，这通常被解释为意味着提供生成结果的原始数据、生成结果的计算机代码以及所需的任何其他信息，如工作流程和输入参数。许多期刊正在修订作者指南，以纳入数据和代码可用性。这项工作评估了期刊政策的有效性，该政策要求作者应要求在出版后提供重复性所需的数据和代码。我们通过(I)向作者索要数据和代码以及(Ii)尝试复制已发表的发现来评估这种政策的有效性。我们随机抽取了2011年2月政策实施后发表在《科学》杂志上的204篇科学论文作为样本。我们发现，我们能够从44%的样本中获得文物，并能够以26%的比例重现这些发现。我们发现这一政策-作者应要求在发布后免除数据和代码-比没有政策有所改善，但目前还不足以重现。
A key component of scientific communication is sufficient information for other researchers in the field to reproduce published findings. For computational and data-enabled research, this has often been interpreted to mean making available the raw data from which results were generated, the computer code that generated the findings, and any additional information needed such as workflows and input parameters. Many journals are revising author guidelines to include data and code availability. This work evaluates the effectiveness of journal policy that requires the data and code necessary for reproducibility be made available postpublication by the authors upon request. We assess the effectiveness of such a policy by (i) requesting data and code from authors and (ii) attempting replication of the published findings. We chose a random sample of 204 scientific papers published in the journal Science after the implementation of their policy in February 2011. We found that we were able to obtain artifacts from 44% of our sample and were able to reproduce the findings for 26%. We find this policy-author remission of data and code postpublication upon request-an improvement over no policy, but currently insufficient for reproducibility.