LATENT VARIABLE GRAPHICAL MODEL SELECTION VIA CONVEX OPTIMIZATION
LATENT VARIABLE GRAPHICAL MODEL SELECTION VIA CONVEX OPTIMIZATION
复制标题
DOI：
10.1214/11-aos949
复制
发表时间：
2012-08-01
影响因子：
4.5
通讯作者：
Willsky, Alan S.
中科院分区：
文献类型：
--
作者：
Chandrasekaran, Venkat;Parrilo, Pablo A.;Willsky, Alan S.
假设我们观察一组随机变量集合的一个子集的样本。没有提供关于潜在变量的数量以及潜在变量和观测变量之间关系的额外信息。是否有可能发现潜在成分的数量，并对整个变量集合学习一个统计模型呢？我们在潜在变量和观测变量联合服从高斯分布，且观测变量以潜在变量为条件的条件统计量由一个图模型指定的设定下解决这个问题。作为第一步，我们给出在仅给定观测变量的边缘统计量的情况下，此类潜在变量高斯图模型可识别的自然条件。本质上，这些条件要求观测变量之间的条件图模型是稀疏的，而潜在变量的影响“分布”在大多数观测变量上。接下来，我们针对这种潜在变量设定提出一个基于正则化最大似然的易处理凸规划用于模型选择；正则化项同时使用了\(l(1)\)范数和核范数。我们的建模框架可以看作是降维（用于识别潜在变量）和图建模（用于捕捉不归因于潜在变量的剩余统计结构）的组合，并且它能一致地估计潜在成分的数量以及观测变量之间的条件图模型结构。这些结果适用于潜在/观测变量的数量随着观测变量的样本数量增长的高维设定。稀疏矩阵和低秩矩阵的代数簇的几何性质在我们的分析中起着重要作用。
Suppose we observe samples of a subset of a collection of random variables. No additional information is provided about the number of latent variables, nor of the relationship between the latent and observed variables. Is it possible to discover the number of latent components, and to learn a statistical model over the entire collection of variables? We address this question in the setting in which the latent and observed variables are jointly Gaussian, with the conditional statistics of the observed variables conditioned on the latent variables being specified by a graphical model. As a first step we give natural conditions under which such latent-variable Gaussian graphical models are identifiable given marginal statistics of only the observed variables. Essentially these conditions require that the conditional graphical model among the observed variables is sparse, while the effect of the latent variables is "spread out" over most of the observed variables. Next we propose a tractable convex program based on regularized maximum-likelihood for model selection in this latent-variable setting; the regularizer uses both the l(1) norm and the nuclear norm. Our modeling framework can be viewed as a combination of dimensionality reduction (to identify latent variables) and graphical modeling (to capture remaining statistical structure not attributable to the latent variables), and it consistently estimates both the number of latent components and the conditional graphical model structure among the observed variables. These results are applicable in the high-dimensional setting in which the number of latent/observed variables grows with the number of samples of the observed variables. The geometric properties of the algebraic varieties of sparse matrices and of low-rank matrices play an important role in our analysis.