Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition.
Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition.
复制标题
DOI：
10.1098/rsos.180448
复制
发表时间：
2018-08
影响因子：
3.5
通讯作者：
Frank MC
中科院分区：
文献类型：
--
作者：
Hardwicke TE;Mathur MB;MacDonald K;Nilsonne G;Banks GC;Kidwell MC;Hofelich Mohr A;Clayton E;Yoon EJ;Henry Tessler M;Lenne RL;Altman S;Long B;Frank MC
获取数据是一个高效、进步和最终自我纠正的科学生态系统的关键特征。但是，数据共享的原则上的好处在实践中实现的程度尚不清楚。重要的是，很大程度上不知道是否可以通过对共享数据进行重复报告分析来复制已发表的研究结果（“分析可复制性”）。为了调查这一点，我们对《认知》杂志上介绍的强制性开放数据政策进行了观察性评估。中断的时间序列分析表明，政策后的数据可用声明大幅增加（104/417，政策前25%至136/174，政策后78%），尽管并非所有数据都可重复使用（23/104，政策前22%至85/136，政策后62%）。对于确定具有可重复使用数据的35篇文章，我们尝试重现1324个目标值。最终，64个值无法在10%的误差范围内重现。对于22篇文章，所有目标值均被复制，但其中11篇需要作者协助。对于13篇文章，尽管有作者协助，但至少有一个值无法复制。重要的是，没有明确的迹象表明原始结论受到严重影响。强制性开放数据政策可以提高数据共享的频率和质量。然而，次优的数据管理，不明确的分析规范和报告错误可能会阻碍分析的可重复性，破坏数据共享的实用性和科学发现的可信度。
Access to data is a critical feature of an efficient, progressive and ultimately self-correcting scientific ecosystem. But the extent to which in-principle benefits of data sharing are realized in practice is unclear. Crucially, it is largely unknown whether published findings can be reproduced by repeating reported analyses upon shared data (‘analytic reproducibility’). To investigate this, we conducted an observational evaluation of a mandatory open data policy introduced at the journal Cognition. Interrupted time-series analyses indicated a substantial post-policy increase in data available statements (104/417, 25% pre-policy to 136/174, 78% post-policy), although not all data appeared reusable (23/104, 22% pre-policy to 85/136, 62%, post-policy). For 35 of the articles determined to have reusable data, we attempted to reproduce 1324 target values. Ultimately, 64 values could not be reproduced within a 10% margin of error. For 22 articles all target values were reproduced, but 11 of these required author assistance. For 13 articles at least one value could not be reproduced despite author assistance. Importantly, there were no clear indications that original conclusions were seriously impacted. Mandatory open data policies can increase the frequency and quality of data sharing. However, suboptimal data curation, unclear analysis specification and reporting errors can impede analytic reproducibility, undermining the utility of data sharing and the credibility of scientific findings.
登录
查看更多内容
影响因子：
17.1
作者：
Goodman, Steven N.;Fanelli, Daniele;Ioannidis, John P. A.
通讯作者：
Ioannidis, John P. A.
影响因子：
4.8
作者：
Evangelou, E;Trikalinos, TA;Ioannidis, JPA
通讯作者：
Ioannidis, JPA
影响因子：
1.5
作者：
McCullough, B. D.;McGeary, Kerry Anne;Harrison, Teresa D.
通讯作者：
Harrison, Teresa D.
影响因子：
1.8
作者：
Marwick, Ben;Boettiger, Carl;Mullen, Lincoln
通讯作者：
Mullen, Lincoln
影响因子：
7.7
作者：
Bernal JL;Cummins S;Gasparrini A
通讯作者：
Gasparrini A