The Importance of Understanding False Discoveries and the Accuracy Paradox When Evaluating Quantitative Studies
The Importance of Understanding False Discoveries and the Accuracy Paradox When Evaluating Quantitative Studies
复制标题
评估定量研究时理解错误发现和准确性悖论的重要性
DOI：
10.22158/sssr.v2n2p1
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Rodney J. Maiden
中科院分区：
文献类型：
--
作者：
K. Davis.;Rodney J. Maiden
虽然零假设显著性检验（NHST）的局限性在心理学文献中有很好的记载，但准确性悖论，它简洁地说明了已发表研究的一个重要局限性，从未被提及。准确性悖论出现时，具有较高准确性的测试正确分类特定结果的工作比具有较低准确性的测试差，这表明依赖准确性作为测试有用性的度量并不总是最好的度量。由于准确性是I型和II型错误率的函数，因此仅仅因为这些错误被最小化而将研究结果解释为准确可能会产生误导。一旦做出了关于统计显著性的决定，I型和II型错误率就不能直接向读者提供信息。相反，在评估研究结果时，错误发现率和错误遗漏率更能提供信息。鉴于文献中普遍存在发表偏倚和小效应量，考虑错误发现的可能性尤为重要。当估计错误发现率时，很容易理解为什么许多心理学研究无法复制。
Although the limitations of null hypothesis significance testing (NHST) are well documented in the psychology literature, the accuracy paradox, which concisely states an important limitation of published research, is never mentioned. The accuracy paradox appears when a test with higher accuracy does a poorer job of correctly classifying a particular outcome than a test with lower accuracy, which suggests that a reliance on accuracy as a metric for a test’s usefulness is not always the best metric. Since accuracy is a function of type I and II error rates, it can be misleading to interpret a study’s results as accurate simply because these errors are minimized. Once a decision has been made regarding statistical significance, type I and II error rates are not directly informative to the reader. Instead, false discovery and false omission rates are more informative when evaluating the results of a study. Given the prevalence of publication bias and small effect sizes in the literature, the possibility of a false discovery is especially important to consider. When false discovery rates are estimated, it is easy to understand why many studies in psychology cannot be replicated.
DOI：
10.1001/jama.1990.03440100097014
发表时间：
1990-03
期刊：
JAMA
影响因子：
--
作者：
K. Dickersin
通讯作者：
K. Dickersin