Dual-Space Analysis of the Sparse Linear Model
Dual-Space Analysis of the Sparse Linear Model
复制标题
DOI：
--
复制
发表时间：
2012-07
期刊：
影响因子：
--
通讯作者：
D. Wipf;Yi Wu
中科院分区：
文献类型：
--
作者：
D. Wipf;Yi Wu
稀疏线性（或广义线性）模型将标准似然函数与关于未知系数的稀疏先验结合联合收割机。这些先验可以方便地表示为具有不同方差超参数的零均值高斯的最大化。标准MAP估计（类型I）涉及最大化超参数和系数，而经验贝叶斯替代方案（类型II）首先将系数边缘化，然后最大化超参数，导致易于处理的后验近似。基础成本函数可以通过Wipf等人（2011）的双空间框架进行关联，该框架允许在系数或超参数空间中表达I型或II型目标。这种观点是有用的，因为一些分析或扩展更有利于在一个或另一个空间的发展。在这里，我们考虑估计的权衡参数平衡稀疏性和数据拟合。由于这个参数实际上是一个方差，自然估计量通过在超参数（方差）空间中评估问题而存在，将自然思想从II型转换为解决I型不太直观的问题。相比之下，对于局部和全局解的更新规则和稀疏性的分析，以及对更一般的似然模型的扩展，我们可以利用为I型开发的系数空间技术，并将其应用于II型。例如，这使我们能够证明，II型启发的技术可以成功地恢复稀疏系数时，不利的限制等距属性（RIP）导致流行的L1重建失败。它也有利于分析类型II时，非高斯似然模型导致棘手的积分。
Sparse linear (or generalized linear) models combine a standard likelihood function with a sparse prior on the unknown coefficients. These priors can conveniently be expressed as a maximization over zero-mean Gaussians with different variance hyperparameters. Standard MAP estimation (Type I) involves maximizing over both the hyperparameters and coefficients, while an empirical Bayesian alternative (Type II) first marginalizes the coefficients and then maximizes over the hyperparameters, leading to a tractable posterior approximation. The underlying cost functions can be related via a dual-space framework from Wipf et al. (2011), which allows both the Type I or Type II objectives to be expressed in either coefficient or hyperparmeter space. This perspective is useful because some analyses or extensions are more conducive to development in one space or the other. Herein we consider the estimation of a trade-off parameter balancing sparsity and data fit. As this parameter is effectively a variance, natural estimators exist by assessing the problem in hyperparameter (variance) space, transitioning natural ideas from Type II to solve what is much less intuitive for Type I. In contrast, for analyses of update rules and sparsity properties of local and global solutions, as well as extensions to more general likelihood models, we can leverage coefficient-space techniques developed for Type I and apply them to Type II. For example, this allows us to prove that Type II-inspired techniques can be successful recovering sparse coefficients when unfavorable restricted isometry properties (RIP) lead to failure of popular L1 reconstructions. It also facilitates the analysis of Type II when non-Gaussian likelihood models lead to intractable integrations.