Testing the reproducibility and robustness of the cancer biology literature by robot.
Testing the reproducibility and robustness of the cancer biology literature by robot.
复制标题
DOI：
10.1098/rsif.2021.0821
复制
发表时间：
2022-04
期刊：
影响因子：
--
通讯作者：
King RD
中科院分区：
文献类型：
--
作者：
Roper K;Abdel-Rehim A;Hubbard S;Carpenter M;Rzhetsky A;Soldatova L;King RD
科学结果不仅应该是“可重复的”（在相同条件下在同一实验室中可复制），而且应该是“可复制的”（在类似条件下在其他实验室中可复制）。如果可能，结果还应该是“稳健的”（在广泛的条件下可复制）。只有一小部分已发表的生物医学结果的重现性和稳健性得到了测试;此外，当重现性测试时，往往找不到。这种情况被称为“重现性危机”，这是生物医学面临的最重要的问题之一。如果能够自动化再现性测试，这一危机将得到解决。在这里，我们描述了从文献中自动提取的关于癌细胞生物学的简单陈述（命题）的再现性和鲁棒性的半自动化测试。从12260篇论文中，我们自动提取了预测描述乳腺癌药物治疗后基因表达变化的实验结果的语句，从中我们选择了74个具有高度生物医学意义的语句。为了测试这些陈述的可重复性，两个不同的团队使用了实验室自动化系统Eve和两种乳腺癌细胞系（MCF 7和MDA-MB-231）。在43个陈述中发现了可重复性的统计学显着证据，在22个陈述中发现了可重复性/稳健性的显着证据。在两个案例中，自动化系统有了意外的发现。再现/强大的知识提供了对癌症的重要见解。我们的结论是，半自动化的再现性测试是目前可以实现的，它可以扩大规模，以产生可靠的知识的实质性来源，自动化有可能减轻再现性危机。
Scientific results should not just be ‘repeatable’ (replicable in the same laboratory under identical conditions), but also ‘reproducible’ (replicable in other laboratories under similar conditions). Results should also, if possible, be ‘robust’ (replicable under a wide range of conditions). The reproducibility and robustness of only a small fraction of published biomedical results has been tested; furthermore, when reproducibility is tested, it is often not found. This situation is termed ‘the reproducibility crisis', and it is one the most important issues facing biomedicine. This crisis would be solved if it were possible to automate reproducibility testing. Here, we describe the semi-automated testing for reproducibility and robustness of simple statements (propositions) about cancer cell biology automatically extracted from the literature. From 12 260 papers, we automatically extracted statements predicted to describe experimental results regarding a change of gene expression in response to drug treatment in breast cancer, from these we selected 74 statements of high biomedical interest. To test the reproducibility of these statements, two different teams used the laboratory automation system Eve and two breast cancer cell lines (MCF7 and MDA-MB-231). Statistically significant evidence for repeatability was found for 43 statements, and significant evidence for reproducibility/robustness in 22 statements. In two cases, the automation made serendipitous discoveries. The reproduced/robust knowledge provides significant insight into cancer. We conclude that semi-automated reproducibility testing is currently achievable, that it could be scaled up to generate a substantive source of reliable knowledge and that automation has the potential to mitigate the reproducibility crisis.
登录
查看更多内容
影响因子：
7.3
作者：
Nelson KM;Dahlin JL;Bisson J;Graham J;Pauli GF;Walters MA
通讯作者：
Walters MA
影响因子：
15.8
作者：
Ioannidis, JPA
通讯作者：
Ioannidis, JPA
姜黄素：通过调节分子途径预防癌症的潜在候选者。
影响因子：
--
作者：
Rahmani AH;Al Zohairy MA;Aly SM;Khan MA
通讯作者：
Khan MA
影响因子：
3.7
作者：
Ko BS;Lu YJ;Yao WL;Liu TA;Tzean SS;Shen TL;Liou JY
通讯作者：
Liou JY
影响因子：
3
作者：
Miwa M;Pyysalo S;Ohta T;Ananiadou S
通讯作者：
Ananiadou S