On “Field Significance” and the False Discovery Rate
On “Field Significance” and the False Discovery Rate
复制标题
DOI：
10.1175/jam2404.1
复制
发表时间：
2006-09
影响因子：
3
通讯作者：
D. Wilks
中科院分区：
文献类型：
--
作者：
D. Wilks
评估多个假设检验的联合统计显著性的传统方法（即，在气象学和气候学中，“场”或“全局”显著性（significance）是对产生名义上显著的结果的个体（或“局部”）测试的数量进行计数，然后在如果所有局部零假设都为真的情况下将发生的这种计数的分布的上下文中判断该整数值的异常性。灵敏度（即，统计功效）潜在地受到检验统计量的离散性质以及该方法忽略局部显著检验拒绝其零假设的置信度的事实的损害。另一个没有这些问题的全局检验统计量是所有局部检验中的最小p值。使用最小局部p值作为全局检验统计量的字段显著性评估，也称为步行者检验，与多个检验的联合评估有很强的联系，以控制“错误发现率”（FDR，或不正确的局部零假设拒绝的预期分数）。特别是，使用最小局部p值来评估全局水平上的字段显著性几乎等同于基于FDR准则的稍微更强大的全局检验。步行者的测试和FDR方法共享的另一个优点是，两者对测试场内的空间依赖性都是鲁棒的。FDR方法不仅提供了比传统计数程序更广泛适用且通常更强大的现场显著性检验，而且还允许更好地识别具有显著差异的位置，因为少于全球100%（平均）的明显显著局部检验将由真实的局部零假设产生。
The conventional approach to evaluating the joint statistical significance of multiple hypothesis tests (i.e., “field,” or “global,” significance) in meteorology and climatology is to count the number of individual (or “local”) tests yielding nominally significant results and then to judge the unusualness of this integer value in the context of the distribution of such counts that would occur if all local null hypotheses were true. The sensitivity (i.e., statistical power) of this approach is potentially compromised both by the discrete nature of the test statistic and by the fact that the approach ignores the confidence with which locally significant tests reject their null hypotheses. An alternative global test statistic that has neither of these problems is the minimum p value among all of the local tests. Evaluation of field significance using the minimum local p value as the global test statistic, which is also known as the Walker test, has strong connections to the joint evaluation of multiple tests in a way that controls the “false discovery rate” (FDR, or the expected fraction of local null hypothesis rejections that are incorrect). In particular, using the minimum local p value to evaluate field significance at a level global is nearly equivalent to the slightly more powerful global test based on the FDR criterion. An additional advantage shared by Walker’s test and the FDR approach is that both are robust to spatial dependence within the field of tests. The FDR method not only provides a more broadly applicable and generally more powerful field significance test than the conventional counting procedure but also allows better identification of locations with significant differences, because fewer than global 100% (on average) of apparently significant local tests will have resulted from local null hypotheses that are true.