STRONG LIMIT THEOREMS ON MODEL SELECTION IN GENERALIZED LINEAR REGRESSION WITH BINOMIAL RESPONSES
STRONG LIMIT THEOREMS ON MODEL SELECTION IN GENERALIZED LINEAR REGRESSION WITH BINOMIAL RESPONSES
复制标题
二项式响应广义线性回归模型选择的强极限定理
作者：
G. Qian;Yuehua Wu
证明了具有二项响应的广义线性回归模型参数的极大似然估计的一个迭代对数律。然后利用这个结果推导出最大对数似然函数与真对数似然函数之差的渐近界。进一步利用该方法建立了基于惩罚似然的模型选择准则的强一致性。我们已经证明，在某些一般条件下，如果惩罚项是模型维数的递增函数，并且阶数在O（log log n）和O(n)之间，则模型选择准则几乎肯定会选择最简单的正确模型。为了说明结果，讨论了涉及常用链接函数的案例。线性回归中的一项重要任务是确定可用解释变量的最佳子集，以形成最佳预测响应变量的模型。我们参考George（2002）和Rao和Wu（2001）对这一研究领域的详细调查。在众多的模型选择方法中，AIC和BIC等经典的模型选择方法仍在实践中得到广泛应用。因此，研究许多问题尚未建立的模型选择准则的渐近性质是很有意义的。本文主要研究具有二项响应的广义线性模型的变量选择问题。我们考虑了一组模型选择准则，如AIC、BIC、Cp和随机复杂性准则，它们遵循惩罚对数似然的形式。我们假设所有影响响应变量的解释变量在观测中都是可用的，因此选择最简单的正确模型是可能的。在某些一般条件下，我们建立了相对于真对数似然函数的最大对数似然函数的强表示。基于这种表示，我们表明，当样本量n突然变大时，几乎肯定会选择最简单的正确模型
We prove a law of iterated logarithm for the maximum likelihood es- timator of the parameters in a generalized linear regression model with binomial response. This result is then used to derive an asymptotic bound for the dierence between the maximum log-likelihood function and the true log-likelihood. It is further used to establish the strong consistency of some penalized likelihood based model selection criteria. We have shown that, under some general conditions, a model selection criterion will select the simplest correct model almost surely if the penalty term is an increasing function of the model dimension and has an order between O(log log n) and O(n). Cases involving the commonly used link functions are discussed for illustration of the results. An important task in linear regression is to identify an optimal subset of available explanatory variables to form a model for best predicting the response variable. We refer to George (2002) and Rao and Wu (2001) for a detailed survey in this area of research. Among the many model selection methods, the classical ones like AIC and BIC are still widely used in practice. It is therefore of interest to investigate the asymptotic properties of model selection criteria which have not yet been established for many problems. In this paper, we focus on variable selection in generalized linear models with binomial responses. We consider a set of model selection criteria, such as AIC, BIC, Cp and the stochastic complexity criterion, that follow the form of a penalized log-likelihood. We assume that all the explanatory variables af- fecting the response variable are available in observations, so that selecting the simplest correct model is possible. We establish a strong representation for the maximum log-likelihood function relative to the true log-likelihood under some general conditions. Based on this representation we show that, when the sample size n is sucien tly large, the simplest correct model is selected almost surely if