Quantifying reproducibility in computational biology: the case of the tuberculosis drugome.
Quantifying reproducibility in computational biology: the case of the tuberculosis drugome.
复制标题
DOI：
10.1371/journal.pone.0080278
复制
发表时间：
2013
期刊：
影响因子：
3.7
通讯作者：
Gil Y
中科院分区：
文献类型：
--
作者：
Garijo D;Kinnings S;Xie L;Xie L;Zhang Y;Bourne PE;Gil Y
重现典型计算生物学论文中的结果有多容易？通过经验或直觉，读者已经知道答案是困难的或根本没有。在本文中，我们试图量化这一困难，复制以前发表的论文，为不同类别的用户（从用户的专业知识不多的领域专家），并建议的方式，这种情况可能会得到改善。量化是通过估计再现原始论文中描述的方法中的每个步骤所需的时间来实现的，并使它们成为再现原始结果的显式工作流程的一部分。重现该方法需要几个月的努力，并且需要使用新版本和新软件，这对重建和验证结果提出了挑战。量化导致“再现性地图”，显示新手研究人员只能再现方法中的几个步骤，只有具有该领域先进知识的专家研究人员才能完整地再现方法。工作流程本身与支持软件和数据一起作为在线资源发布。本文最后简要讨论了要求再现性的成本与效益方面的复杂性，并与我们的意见和指导方针，以提高再现性的desiderata。这不仅意味着从已发表的论文中复制他人的工作，而且还意味着复制自己实验室的工作。
How easy is it to reproduce the results found in a typical computational biology paper? Either through experience or intuition the reader will already know that the answer is with difficulty or not at all. In this paper we attempt to quantify this difficulty by reproducing a previously published paper for different classes of users (ranging from users with little expertise to domain experts) and suggest ways in which the situation might be improved. Quantification is achieved by estimating the time required to reproduce each of the steps in the method described in the original paper and make them part of an explicit workflow that reproduces the original results. Reproducing the method took several months of effort, and required using new versions and new software that posed challenges to reconstructing and validating the results. The quantification leads to “reproducibility maps” that reveal that novice researchers would only be able to reproduce a few of the steps in the method, and that only expert researchers with advance knowledge of the domain would be able to reproduce the method in its entirety. The workflow itself is published as an online resource together with supporting software and data. The paper concludes with a brief discussion of the complexities of requiring reproducibility in terms of cost versus benefit, and a desiderata with our observations and guidelines for improving reproducibility. This has implications not only in reproducing the work of others from published papers, but reproducing work from one’s own laboratory.
登录
查看更多内容
影响因子：
4.3
作者：
Kinnings SL;Xie L;Fung KH;Jackson RM;Xie L;Bourne PE
通讯作者：
Bourne PE
影响因子：
12.3
作者：
Goecks J;Nekrutenko A;Taylor J;Galaxy Team
通讯作者：
Galaxy Team
影响因子：
1.8
作者：
Decullier, Evelyne;Huot, Laure;Maisonneuve, Herve
通讯作者：
Maisonneuve, Herve
影响因子：
48
作者：
Bell, Alexander W.;Deutsch, Eric W.;Au, Catherine E.;Kearney, Robert E.;Beavis, Ron;Sechi, Salvatore;Nilsson, Tommy;Bergeron, John J. M.
通讯作者：
Bergeron, John J. M.
影响因子：
1.1
作者：
Bonnet, Philippe;Manegold, Stefan;Shasha, Dennis
通讯作者：
Shasha, Dennis