An accurate comparison of methods for quantifying variable importance in artificial neural networks using simulated data
An accurate comparison of methods for quantifying variable importance in artificial neural networks using simulated data
复制标题
DOI：
10.1016/j.ecolmodel.2004.03.013
复制
发表时间：
2004-11-01
影响因子：
3.1
通讯作者：
Death, RG
中科院分区：
文献类型：
--
作者：
Olden, JD;Joy, MK;Death, RG
人工神经网络（ANN）作为一种强大的统计建模技术，在生态科学中受到越来越多的关注;然而，它们也被贴上了“黑匣子”的标签，因为它们被认为对预测过程中自变量的贡献提供了很少的解释性见解。最近发表在《生态建模》上的一篇论文[回顾和比较研究人工神经网络模型中变量贡献的方法，生态模型。160（2003）249-264]通过提供用于估计生态学中常用的神经网络中的变量重要性的八种不同方法的综合比较来解决这个问题。不幸的是，不同方法的比较是基于经验数据集，这排除了建立关于不同方法的真正准确性和精确性的概括的能力，因为变量的真正重要性是未知的。在这里，我们提供了一个更合适的比较不同的方法，通过使用Monte Carlo模拟与数据展示定义（因此已知）的数字关系。我们的研究结果表明，一个连接权重的方法，使用原始的输入隐藏和隐藏的输出连接权重的神经网络提供了最好的方法，准确量化变量的重要性，应该比其他常用的方法在生态文献中的青睐。使用这种方法的真实和估计的排名变量的重要性之间的平均相似性为0.92，而其他方法的相似性系数在0.28和0.74之间。此外，连接权重方法是唯一一种始终识别所有预测变量的正确重要性排名的方法，而其他方法要么只识别网络中的前几个重要变量，要么根本不识别变量。最显著的结果是，加尔森的算法是最差的性能的方法，但最常用的生态文学。总之，本研究提供了一个强大的比较不同的方法来评估变量的重要性，神经网络，可以推广到其他数据，并从有效的建议，可以为未来的研究。(C)2004 Elsevier B. V.保留所有权利。
Artificial neural networks (ANNs) are receiving greater attention in the ecological sciences as a powerful statistical modeling technique; however, they have also been labeled a "black box" because they are believed to provide little explanatory insight into the contributions of the independent variables in the prediction process. A recent paper published in Ecological Modelling [Review and comparison of methods to study the contribution of variables in artificial neural network models, Ecol. Model. 160 (2003) 249-264] addressed this concern by providing a comprehensive comparison of eight different methodologies for estimating variable importance in neural networks that are commonly used in ecology. Unfortunately, comparisons of the different methodologies were based on an empirical dataset, which precludes the ability to establish generalizations regarding the true accuracy and precision of the different approaches because the true importance of the variables is unknown. Here, we provide a more appropriate comparison of the different methodologies by using Monte Carlo simulations with data exhibiting defined (and consequently known) numeric relationships. Our results show that a Connection Weight Approach that uses raw input-hidden and hidden-output connection weights in the neural network provides the best methodology for accurately quantifying variable importance and should be favored over the other approaches commonly used in the ecological literature. Average similarity between true and estimated ranked variable importance using this approach was 0.92, whereas, similarity coefficients ranged between 0.28 and 0.74 for the other approaches. Furthermore, the Connection Weight Approach was the only method that consistently identified the correct ranked importance of all predictor variables, whereas, the other methods either only identified the first few important variables in the network or no variables at all. The most notably result was that Garson's Algorithm was the poorest performing approach, yet is the most commonly used in the ecological literature. In conclusion, this study provides a robust comparison of different methodologies for assessing variable importance in neural networks that can be generalized to other data and from which valid recommendations can be made for future studies. (C) 2004 Elsevier B.V. All rights reserved.