The Harm Done to Reproducibility by the Culture of Null Hypothesis Significance Testing
The Harm Done to Reproducibility by the Culture of Null Hypothesis Significance Testing
复制标题
DOI：
10.1093/aje/kwx261
复制
发表时间：
2017-09-15
影响因子：
5
通讯作者：
Lash, Timothy L.
中科院分区：
文献类型：
--
作者：
Lash, Timothy L.
在过去几年中，科学界的利益相关者对科学结果缺乏可重复性提出了警告。作为回应，颁布了期刊指南，并要求资助申请人解决其拟议项目的严谨性和可重复性。这两种解决方案都没有解决罪魁祸首，即主导统计分析和推理的零假设显着性检验文化。在创新型研究企业中，基于原假设显着性检验选择进一步评估的结果注定会产生低比例的可重复结果和高比例的最初被高估的效应。此外，零假设显着性检验的文化不鼓励进行定量调整以解释系统误差和先验信息的定量结合。否则，这些策略将提高可重复性，并且之前在有关该主题的广泛引用的文献中并未提出过。在不放弃零假设显着性检验文化并实施这些统计分析和推理的替代方法的情况下，所有其他提高可重复性的策略最多只能产生边际收益。
In the last few years, stakeholders in the scientific community have raised alarms about a perceived lack of reproducibility of scientific results. In reaction, guidelines for journals have been promulgated and grant applicants have been asked to address the rigor and reproducibility of their proposed projects. Neither solution addresses a primary culprit, which is the culture of null hypothesis significance testing that dominates statistical analysis and inference. In an innovative research enterprise, selection of results for further evaluation based on null hypothesis significance testing is doomed to yield a low proportion of reproducible results and a high proportion of effects that are initially overestimated. In addition, the culture of null hypothesis significance testing discourages quantitative adjustments to account for systematic errors and quantitative incorporation of prior information. These strategies would otherwise improve reproducibility and have not been previously proposed in the widely cited literature on this topic. Without discarding the culture of null hypothesis significance testing and implementing these alternative methods for statistical analysis and inference, all other strategies for improving reproducibility will yield marginal gains at best.