MLE convergence speed to information projection of exponential family: Criterion for model dimension and sample size -- complete proof version--
MLE convergence speed to information projection of exponential family: Criterion for model dimension and sample size -- complete proof version--
复制标题
指数族信息投影的 MLE 收敛速度：模型维数和样本量的判据——完整证明版本——
DOI：
--
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Y. Sheena
中科院分区：
文献类型：
--
作者：
Y. Sheena
对于分布的参数模型，考虑模型中最接近模型外真实分布的分布。用Kullback-Leibler （KL）散度测量两个分布之间的接近度，最接近的分布称为“信息投影”。最大似然估计量（MLE）的估计风险定义为信息投影与插入最大似然估计量的预测分布之间K-L散度的期望。在此，导出了风险的渐近展开式直至n阶，并研究了真实分布与信息投影之间的贝叶斯误差率小于规定值的风险的充分条件。结合这些结果，提出了“p−n准则”，该准则决定了MLE是否足够接近给定模型和样本的信息投影。特别是，指数族模型的判据相对简单，可以用于没有显式归一化常数形式的复杂模型。这个准则可以构成样本大小或模型可接受性问题的解决方案。使用的p−n标准演示了两个实际的数据集。研究了结果与信息准则之间的关系。
For a parametric model of distributions, the closest distribution in the model to the true distribution located outside the model is considered. Measuring the closeness between two distributions with the Kullback-Leibler (KL) divergence, the closest distribution is called the “information projection.” The estimation risk of the maximum likelihood estimator (MLE) is defined as the expectation of K-L divergence between the information projection and the predictive distribution with plugged-in MLE. Here, the asymptotic expansion of the risk is derived up to n-order, and the sufficient condition on the risk for the Bayes error rate between the true distribution and the information projection to be lower than a specified value is investigated. Combining these results, the “p−n criterion” is proposed, which determines whether the MLE is sufficiently close to the information projection for the given model and sample. In particular, the criterion for an exponential family model is relatively simple and can be used for a complex model with no explicit form of normalizing constant. This criterion can constitute a solution to the sample size or model acceptance problem. Use of the p − n criteria is demonstrated for two practical datasets. The relationship between the results and information criteria is also studied.
影响因子：
32.8
作者：
Wainwright, Martin J.;Jordan, Michael I.
通讯作者：
Jordan, Michael I.