BOOTSTRAP ESTIMATE OF KULLBACK-LEIBLER INFORMATION FOR MODEL SELECTION
BOOTSTRAP ESTIMATE OF KULLBACK-LEIBLER INFORMATION FOR MODEL SELECTION
复制标题
用于模型选择的 KULLBACK-LEIBLER 信息的自举估计
DOI：
--
复制
发表时间：
1997
期刊：
影响因子：
--
通讯作者：
R. Shibata
中科院分区：
文献类型：
--
作者：
R. Shibata
Kullback-Leibler信息的估计是推导统计模型选择过程的关键部分，该过程与AIC一样，是基于似然原理的。为了区分嵌套模型，我们必须将Kullback-Leibler信息估计到一个常数的数量级，而Kullback-Leibler信息本身是观测值数量的数量级。AIC中使用的修正项是如何满足这一要求的一个例子；然而，修正是对对数最大似然的简单的偏差修正，并且不能保证这种偏差修正产生对Kullback-Leibler信息的良好估计。本文研究Kullback-Leibler信息的Bootstrap型估计作为另一种选择。我们首先证明了Efron(1983,1986)和Cavanaugh和Shumway(1997)提出的Bootstrap估计至少是渐近等价的，并且存在许多其他等价的Bootstrap估计。我们还证明了所有这些方法都渐近等价于非Bootstrap方法TIC(Takeuchi(1976))，TIC是重抽样方法非参数时AIC的推广。否则，例如，如果重采样方法是参数的，则它们与AIC渐近等价。因此，如果有足够的观测可用并且非自举估计AIC或TIC的简单计算不是负担，则使用自举类型估计是不有利的。同时，如果非引导估计过于复杂而无法进行分析评估，那么使用引导估计来代替非引导估计也是合理和有利的。
Estimation of Kullback-Leibler information is a crucial part of deriving a statistical model selection procedure which, like AIC, is based on the likelihood principle. To discriminate between nested models, we have to estimate Kullback- Leibler information up to the order of a constant, while Kullback-Leibler informa- tion itself is of the order of the number of observations. A correction term employed in AIC is an example of how to fulfill this requirement; however the correction is a simple minded bias correction to the log maximum likelihood and there is no assurance that such a bias correction yields a good estimate of Kullback-Leibler information. In this paper we investigate a bootstrap type estimate of Kullback- Leibler information as an alternative. We first show that both bootstrap estimates proposed by Efron (1983, 1986) and by Cavanaugh and Shumway (1997) are at least asymptotically equivalent and there exist many other equivalent bootstrap estimates. We also show that all such methods are asymptotically equivalent to a non-bootstrap method known as TIC(Takeuchi (1976)), which is a generalization of AIC when the re-sampling method is non-parametric. Otherwise, for example, if the re-sampling method is parametric they are asymptotically equivalent to AIC. Therefore, the use of a bootstrap type estimate is not advantageous if enough ob- servations are available and simple calculations of a non-bootstrap estimate AIC or TIC is not a burden. At the same time, it is also true that the use of a bootstrap estimate in place of a non-bootstrap estimate is reasonable and advantageous if the non-bootstrap estimate is too complicated to evaluate analytically.