Multiplicity Eludes Peer Review: The Case of COVID-19 Research.
Multiplicity Eludes Peer Review: The Case of COVID-19 Research.
复制标题
DOI：
10.3390/ijerph18179304
复制
发表时间：
2021-09-03
影响因子：
--
通讯作者：
García LV
中科院分区：
文献类型：
--
作者：
Gutiérrez-Hernández O;García LV
当数据分析涉及多个同时推理时，多重性就会出现，从而增加了虚假发现的机会。这是一个经常被研究人员忽视的问题。在本文中，我们对Web of Science数据库中的COVID-19观察性研究进行了探索性分析。我们基于p值检查了100篇引用率最高的COVID-19同行评审文章，包括多达7100个同时测试，其中50%包括>34个测试，20% > 100个测试。我们发现，进行的检验次数越多，显著结果的数量就越多（r = 0.87，p < 10−6）。摘要中的p值数量与论文中的p值数量无关。然而，摘要中的高度显著性结果（p < 0.001）与论文中p < 0.001显著性的数量密切相关（r = 0.61，p < 10−6）。此外，摘要包括更高比例的显著结果（0.91 vs. 0.50），80%仅报告了显著结果。只有一篇被审查的论文讨论了多重性引起的I型错误膨胀，指出了绕过同行审查过程的潜在虚假结果。我们的结论是，需要特别注意观察性研究中错误发现的可能性增加，包括具有潜在巨大社会影响的不可复制的惊人发现。我们提出了一些易于实施的措施来评估和限制多重性的影响。
Multiplicity arises when data analysis involves multiple simultaneous inferences, increasing the chance of spurious findings. It is a widespread problem frequently ignored by researchers. In this paper, we perform an exploratory analysis of the Web of Science database for COVID-19 observational studies. We examined 100 top-cited COVID-19 peer-reviewed articles based on p-values, including up to 7100 simultaneous tests, with 50% including >34 tests, and 20% > 100 tests. We found that the larger the number of tests performed, the larger the number of significant results (r = 0.87, p < 10−6). The number of p-values in the abstracts was not related to the number of p-values in the papers. However, the highly significant results (p < 0.001) in the abstracts were strongly correlated (r = 0.61, p < 10−6) with the number of p < 0.001 significances in the papers. Furthermore, the abstracts included a higher proportion of significant results (0.91 vs. 0.50), and 80% reported only significant results. Only one reviewed paper addressed multiplicity-induced type I error inflation, pointing to potentially spurious results bypassing the peer-review process. We conclude the need to pay special attention to the increased chance of false discoveries in observational studies, including non-replicated striking discoveries with a potentially large social impact. We propose some easy-to-implement measures to assess and limit the effects of multiplicity.
登录
查看更多内容
关于SARS-COV-2的生物气候相关模型的有用性。
影响因子：
8.3
作者：
Gutiérrez-Hernández O;García LV
通讯作者：
García LV
影响因子：
15.8
作者：
Ioannidis, JPA
通讯作者：
Ioannidis, JPA
DOI：
10.1111/j.2517-6161.1995.tb02031.x
发表时间：
1995-01-01
影响因子：
5.8
作者：
BENJAMINI, Y;HOCHBERG, Y
通讯作者：
HOCHBERG, Y
影响因子：
6.3
作者：
Barua, Zapan;Barua, Sajib;Li, Mingze
通讯作者：
Li, Mingze
影响因子：
3.8
作者：
Lep, Zan;Babnik, Katarina;Hacin Beyazoglu, Kaja
通讯作者：
Hacin Beyazoglu, Kaja