A new criterion for selecting models from partially observed data
A new criterion for selecting models from partially observed data
复制标题
从部分观测数据中选择模型的新标准
DOI：
10.1007/978-1-4612-2660-4_3
复制
发表时间：
1994
影响因子：
1.8
通讯作者：
Hidetoshi Shimodaira
中科院分区：
文献类型：
--
作者：
Hidetoshi Shimodaira
提出了一种用于从部分观测数据中选择统计模型的新准则PDIO(间接观测模型的预测散度)。PDIO是为“间接观测模型”设计的，在该模型中，观测值只能通过随机变量间接获得。也就是说，假设一些潜在的隐藏结构来生成清单变量。例如，无监督学习识别系统、聚类、潜在结构分析、混合分布模型、缺失数据、噪声观测等，或者其最大似然估计器基于EM(期望最大化)算法的模型。PDIO是AIC(Akaike信息准则)的自然扩展，当有直接观测时，这两个准则是等价的。这两个标准都表示为两个项的总和：第一项表示模型对观测数据的拟合程度，第二项表示模型的复杂性。在这两个标准中，拟合度项是等价的，但复杂性项是不同的。复杂性项是模型结构和样本数量的函数，并且被添加是为了考虑观测数据的可靠性。在PDIO中使用估计的真实分布的平均波动作为模型的复杂性。因此，“模型流形”和“观测流形”的相对关系从信息几何的角度反映在PDIO的复杂性项中，而在AIC中则归结为参数的个数。PDIO在“积极地”处理不可观察到的基础结构方面非常独特。本文利用两个Fisher信息矩阵给出了PDIO的一般表达式。利用EM迭代法给出了PDIO的一种近似计算方法。一些计算机模拟显示了该判据是如何工作的。
A new criterion PDIO (predictive divergence for indirect observation models) is proposed for selecting statistical models from partially observed data. PDIO is devised for “indirect observation models”, in which observations are only available indirectly through random variables. That is, some underlying hidden structure is assumed to generate the manifest variables. For example, unsupervised learning recognition systems, clustering, latent structure analysis, mixture distribution models, missing data, noisy observations, etc., or the models whose maximum likelihood estimator is based on the EM (expectation-maximization) algorithm. PDIO is a natural extension of AIC (Akaike’s information criterion), and the two criteria are equivalent when direct observations are available. Both criteria are expressed as the sum of two terms: the first term represents the goodness of fit of the model to the observed data, and the second term represents the model complexity. The goodness ot fit terms are equivalent in both criteria, but the complexity terms are different. The complexity term is a function of model structure and the number of samples and is added in order to take into account the reliability of the observed data. A mean fluctuation of the estimated true distribution is used as the model complexity in PDIO. The relative relation of the “model manifold” and the “observed manifold” is, therefore, reflected in the complexity term of PDIO from the information geometric point of view, whereas it reduces to the number of parameters in AIC. PDIO is very unique in dealing with the unobservable underlying structure “positively.” In this paper the generalized expression of PDIO is shown using two Fisher information matrices. An approximated computation method for PDIO is also presented utilizing EM iterates. Some computer simulations are shown to demonstrate how this criterion works.