Asymptotic Log-Loss of Prequential Maximum Likelihood Codes
Asymptotic Log-Loss of Prequential Maximum Likelihood Codes
复制标题
前置最大似然码的渐近对数损失
DOI：
10.1007/11503415_44
复制
发表时间：
2005
期刊：
影响因子：
--
通讯作者：
S. D. Rooij
中科院分区：
文献类型：
--
作者：
P. Grünwald;S. D. Rooij
我们分析了Dawid-Rissanen序极大似然码相对于单参数指数族模型M。如果数据是i.i.d.根据（基本上）任意的P，则冗余以1/2 c ln n的速率增长。我们证明了c = σ 2 1 /σ 2 2，其中σ 2 1是P的方差，σ 2 2是在KL散度中最接近P的分布M* ∈ M的方差.这表明，前置码的行为完全不同于其他重要的通用码，如2部分MDL，Shtarkov和贝叶斯码，其中c = 1。这种行为在MDL模型选择设置中是不希望出现的。
We analyze the Dawid-Rissanen prequential maximum likelihood codes relative to one-parameter exponential family models M. If data are i.i.d. according to an (essentially) arbitrary P, then the redundancy grows at rate 1/2 c ln n. We show that c = σ 2 1 /σ 2 2 , where σ 2 1 is the variance of P, and σ 2 2 is the variance of the distribution M* ∈ M that is closest to P in KL divergence. This shows that prequential codes behave quite differently from other important universal codes such as the 2-part MDL, Shtarkov and Bayes codes, for which c = 1. This behavior is undesirable in an MDL model selection setting.