A farewell to the sum of Akaike weights: The benefits of alternative metrics for variable importance estimations in model selection
A farewell to the sum of Akaike weights: The benefits of alternative metrics for variable importance estimations in model selection
复制标题
DOI：
10.1111/2041-210x.12835
复制
发表时间：
2017-12-01
影响因子：
6.6
通讯作者：
Dechaume-Moncharmont, Francois-Xavier
中科院分区：
文献类型：
--
作者：
Galipaud, Matthias;Gillingham, Mark A. F.;Dechaume-Moncharmont, Francois-Xavier
1.在前一篇文章中，我们反对在信息论(IT)统计分析中使用Akaike权重之和(Sw)作为区分真假变量的度量。最近的一篇文章(Giam&Olden，《生态学和进化的方法》，2016，7,388)批评了我们的发现，转而支持西南大学。它指出，(1)我们执行了有偏见的数据生成程序，(2)我们错误地评估了Sw在估计变量解释的数据中的方差比例的能力。我们在这里回应这些观点。吉亚姆和奥登的第一个担忧是没有根据的。在使用他们提出的数据生成代码时，sw仍然非常不精确。为了回应他们的第二个担忧，我们首先列出了变量在IT上下文中的重要性所具有的含义。虽然Sw在方法论教科书中被描述为变量相对重要性的估计(即变量的重要性排名或其对数据方差的相对贡献)，但它也被用作变量绝对重要性的度量(即变量的绝对效应大小或其统计显著性)。然后，我们比较软件和替代指标的能力，以估计变量的绝对或相对重要性。SW值在不同分析中的重复性较低。因此，基于Sw，很难区分弱影响变量和大影响变量。对于变量绝对重要性的估计，实验者应该更喜欢模型平均参数估计和/或基于证据比率的嵌套模型比较。Akaike权重之和也是一个相对重要性可变的较差度量。我们发现，当使用模型平均标准化参数估计时，正确的变量重要性排名通常比使用SW4时更频繁。为了避免生态学和进化论中反复出现的错误，我们因此警告不要使用SW值来估计变量的绝对和相对重要性，我们建议实验者应该转而使用模型平均的标准化参数估计来进行统计推断。
1. In a previous article, we advocated against using the sum of Akaike weights (SW) as a metric to distinguish between genuine and spurious variables in Information Theoretic (IT) statistical analyses. A recent article (Giam & Olden, Methods in Ecology and Evolution, 2016, 7, 388) criticises our finding and instead argues in favour of SW. It points out that (1) we performed a biased data-generation procedure and (2) we erroneously evaluated SW on its capacity to estimate the proportion of variance in the data explained by a variable. We here respond to these points.2. Giam and Olden's first concern is unfounded. When using the data-generating code they proposed, SW remains very imprecise. To respond to their second concern, we first list the meanings taken by a variable's importance in the context of IT. Although, SW is presented as an estimate of variable relative importance in methodological textbooks (i.e. a variable's rank in importance or its relative contribution to the variance in the data), it is also used as a metric of variable absolute importance (i.e. a variable's absolute effect size or its statistical significance). We then compare SW to alternative metrics on its ability to estimate variable absolute or relative importance.3. SW values have low repeatability across analyses. As a result, based on SW, it is hard to distinguish between variables with weak and large effects. For estimations of variable absolute importance, experimenters should prefer model-averaged parameter estimates and/or compare nested models based on evidence ratios. Sum of Akaike weights is also a poor metric of variable relative importance. We showed that correct variable ranking in importance was generally more frequent when using model-averaged standardised parameter estimates, than when using SW.4. To avoid recurrent errors in ecology and evolution, we therefore warn against the use of SW for estimations of variable absolute and relative importance and we propose that experimenters should instead use model-averaged standardised parameter estimates for statistical inferences.