An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014-2017)
An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014-2017)
复制标题
DOI：
10.1098/rsos.190806
复制
发表时间：
2020-02-19
影响因子：
3.5
通讯作者：
Ioannidis, John P. A.
中科院分区：
文献类型：
--
作者：
Hardwicke, Tom E.;Wallach, Joshua D.;Ioannidis, John P. A.
对研究质量的严重担忧催生了一系列旨在提高透明度和可重复性的改革举措，从而促进自我纠正、提高效率和增强研究可信度。元研究评估了一些个人举措的优点；然而，这可能无法反映反映这些努力的累积贡献的更广泛趋势。在这项研究中，我们手动检查了 250 篇文章的随机样本，以估计 2014 年至 2017 年间发表的社会科学文献中一系列透明度和可重复性相关指标的普遍程度。很少有文章表明材料的可用性（16/151，11％[95％置信区间，7％至 16％]），协议（0/156，0％[0％至 1％]），原始数据（11/156，7% [2% 至 13%]）或分析脚本（2/156，1% [0% 至 3%]），并且没有预先注册研究（0/156，0% [0% 至 1%]）。一些文章明确披露了资金来源（或缺乏；74/236，31% [25% 至 37%]），一些文章宣称不存在利益冲突（36/236，15% [11% 至 20%]）。复制研究很少（2/156，1% [0% 至 3%]）。很少有研究通过系统评价（17/151，11% [7% 至 16%]）或荟萃分析（2/151，1% [0% 至 3%]）纳入证据合成。不到一半的文章是公开的（101/250，40% [34% 到 47%]）。最低限度地采用与透明度和可重复性相关的研究实践可能会损害社会科学研究的可信度和效率。本研究建立了一个基线，可以在将来重新审视以评估进展。
Serious concerns about research quality have catalysed a number of reform initiatives intended to improve transparency and reproducibility and thus facilitate self-correction, increase efficiency and enhance research credibility. Meta-research has evaluated the merits of some individual initiatives; however, this may not capture broader trends reflecting the cumulative contribution of these efforts. In this study, we manually examined a random sample of 250 articles in order to estimate the prevalence of a range of transparency and reproducibility-related indicators in the social sciences literature published between 2014 and 2017. Few articles indicated availability of materials (16/151, 11% [95% confidence interval, 7% to 16%]), protocols (0/156, 0% [0% to 1%]), raw data (11/156, 7% [2% to 13%]) or analysis scripts (2/156, 1% [0% to 3%]), and no studies were pre-registered (0/156, 0% [0% to 1%]). Some articles explicitly disclosed funding sources (or lack of; 74/236, 31% [25% to 37%]) and some declared no conflicts of interest (36/236, 15% [11% to 20%]). Replication studies were rare (2/156, 1% [0% to 3%]). Few studies were included in evidence synthesis via systematic review (17/151, 11% [7% to 16%]) or meta-analysis (2/151, 1% [0% to 3%]). Less than half the articles were publicly available (101/250, 40% [34% to 47%]). Minimal adoption of transparency and reproducibility-related research practices could be undermining the credibility and efficiency of social science research. The present study establishes a baseline that can be revisited in the future to assess progress.