Evaluation of excess significance bias in animal studies of neurological diseases.
Evaluation of excess significance bias in animal studies of neurological diseases.
复制标题
DOI：
10.1371/journal.pbio.1001609
复制
发表时间：
2013-07
期刊：
影响因子：
9.8
通讯作者：
Ioannidis JP
中科院分区：
文献类型：
--
作者：
Tsilidis KK;Panagiotou OA;Sena ES;Aretouli E;Evangelou E;Howells DW;Al-Shahi Salman R;Macleod MR;Ioannidis JP
对160项关于神经系统疾病潜在治疗方法的动物研究的荟萃分析的评估显示，统计上显著的结果数量太大而不真实，这表明存在偏差。动物研究产生了有价值的假设，从而导致进行预防性或治疗性临床试验。我们评估了在神经系统疾病的动物研究结果中是否存在统计学显著性的证据，这表明存在偏差。我们使用的数据来自于《实验研究中动物数据的荟萃分析和回顾的协作方法》（CAMARADES）中对干预措施的荟萃分析。根据每个研究在不同合理效应大小假设下的统计能力，比较具有统计显著结果的观察研究数(O)与期望研究数(E)。我们评估了来自160项荟萃分析的4,445个数据集，这些荟萃分析包括阿尔茨海默病（n = 2）、实验性自身免疫性脑脊髓炎（n = 34）、局灶性缺血（n = 16）、脑出血（n = 61）、帕金森病（n = 45）和脊髓损伤（n = 2）。112项荟萃分析（70%）发现名义上（p≤0.05）有统计学意义的总结固定效应。假设最精确研究中的效应大小是可信的效应，4,445个名义上显著的结果中有919个是预期的，而观察到的结果为1,719个（p<10−9）。在所有神经系统疾病中，在由方法学特征定义的所有亚组中，以及根据其他似是而非的效应，都存在过度显著性。在74项（46%）荟萃分析中，不对称测试也显示了小型研究效应的证据。在8项（5%）荟萃分析中，对500多只动物进行了显著有效的干预，没有发现任何偏见迹象。总的来说，在神经系统疾病的文献中，有太多的动物研究具有统计显著的结果。这一观察结果表明存在强烈的偏差，选择性分析和结果报告偏差是合理的解释，并为这些偏差如何影响神经学动物文献的整个研究领域提供了新的证据。研究表明，动物生物医学实验的结果不能转化为人体临床试验；这可能归因于人和动物之间潜在生物学的真正差异，实验设计的缺陷，或动物研究结果报告的偏见。我们使用统计技术来评估发表的具有“阳性”（统计显著）结果的动物研究的数量是否太大而不真实。我们评估了4,445项动物研究，用于160种神经系统疾病的候选治疗方法，并观察到其中1,719项具有“积极”结果，而只有919项研究可以先验地期望获得这样的结果。根据我们的方法，160种被评估的治疗方法中只有8种随后应该在人类身上进行测试。综上所述，我们判断神经系统疾病文献中有太多的动物研究具有“积极”的结果，我们讨论了这一现象的原因和潜在的补救措施。
The evaluation of 160 meta-analyses of animal studies on potential treatments for neurological disorders reveals that the number of statistically significant results was too large to be true, suggesting biases. Animal studies generate valuable hypotheses that lead to the conduct of preventive or therapeutic clinical trials. We assessed whether there is evidence for excess statistical significance in results of animal studies on neurological disorders, suggesting biases. We used data from meta-analyses of interventions deposited in Collaborative Approach to Meta-Analysis and Review of Animal Data in Experimental Studies (CAMARADES). The number of observed studies with statistically significant results (O) was compared with the expected number (E), based on the statistical power of each study under different assumptions for the plausible effect size. We assessed 4,445 datasets synthesized in 160 meta-analyses on Alzheimer disease (n = 2), experimental autoimmune encephalomyelitis (n = 34), focal ischemia (n = 16), intracerebral hemorrhage (n = 61), Parkinson disease (n = 45), and spinal cord injury (n = 2). 112 meta-analyses (70%) found nominally (p≤0.05) statistically significant summary fixed effects. Assuming the effect size in the most precise study to be a plausible effect, 919 out of 4,445 nominally significant results were expected versus 1,719 observed (p<10−9). Excess significance was present across all neurological disorders, in all subgroups defined by methodological characteristics, and also according to alternative plausible effects. Asymmetry tests also showed evidence of small-study effects in 74 (46%) meta-analyses. Significantly effective interventions with more than 500 animals, and no hints of bias were seen in eight (5%) meta-analyses. Overall, there are too many animal studies with statistically significant results in the literature of neurological disorders. This observation suggests strong biases, with selective analysis and outcome reporting biases being plausible explanations, and provides novel evidence on how these biases might influence the whole research domain of neurological animal literature. Studies have shown that the results of animal biomedical experiments fail to translate into human clinical trials; this could be attributed either to real differences in the underlying biology between humans and animals, to shortcomings in the experimental design, or to bias in the reporting of results from the animal studies. We use a statistical technique to evaluate whether the number of published animal studies with “positive” (statistically significant) results is too large to be true. We assess 4,445 animal studies for 160 candidate treatments of neurological disorders, and observe that 1,719 of them have a “positive” result, whereas only 919 studies would a priori be expected to have such a result. According to our methodology, only eight of the 160 evaluated treatments should have been subsequently tested in humans. In summary, we judge that there are too many animal studies with “positive” results in the neurological disorder literature, and we discuss the reasons and potential remedies for this phenomenon.
登录
查看更多内容
影响因子：
2
作者：
Heller, Glenn
通讯作者：
Heller, Glenn
DOI：
10.3758/bf03203630
发表时间：
1996-02-01
期刊：
BEHAVIOR RESEARCH METHODS INSTRUMENTS & COMPUTERS
影响因子：
--
作者：
Erdfelder, E;Faul, F;Buchner, A
通讯作者：
Buchner, A
影响因子：
9.9
作者：
Freedman, M. S.;Bar-Or, A.;Verco, T.
通讯作者：
Verco, T.
影响因子：
4.4
作者：
Bebarta, V;Luyten, D;Heard, K
通讯作者：
Heard, K
影响因子：
8.3
作者：
England, Timothy J.;Abaei, Maryam;Bath, Philip M. W.
通讯作者：
Bath, Philip M. W.