Compressible Distributions for High-Dimensional Statistics
Compressible Distributions for High-Dimensional Statistics
复制标题
DOI：
10.1109/tit.2012.2197174
复制
发表时间：
2012-08-01
影响因子：
2.5
通讯作者：
Davies, Mike E.
中科院分区：
文献类型：
--
作者：
Gribonval, Remi;Cevher, Volkan;Davies, Mike E.
我们开发了一种原则性的方法来识别概率分布，其独立和同分布的实现是可压缩的，即，可以很好地近似为稀疏。我们专注于高斯压缩感知，欠定线性回归的一个例子，其中压缩性是已知的，以确保成功的估计利用稀疏正则化。我们证明了许多分布围绕最大后验概率（MAP）解释稀疏正则估计实际上是不可压缩的，在大的问题大小的限制。我们特别强调了拉普拉斯分布和正则化估计，如Lasso和基追踪去噪。我们严格反驳的神话，压缩感知图像重建的最小化的成功是一个简单的推论的拉普拉斯模型的图像结合贝叶斯MAP估计，并表明，事实上是完全相反的。为了建立这一结果，我们确定非平凡的欠采样区域，简单的最小二乘解决方案几乎肯定优于甲骨文稀疏的解决方案，当数据是从拉普拉斯分布。我们还提供了简单的经验法则来描述类的可压缩和不可压缩分布的基础上，他们的第二和第四时刻。广义高斯分布和广义帕累托分布作为运行的例子。
We develop a principled way of identifying probability distributions whose independent and identically distributed realizations are compressible, i.e., can be well approximated as sparse. We focus on Gaussian compressed sensing, an example of underdetermined linear regression, where compressibility is known to ensure the success of estimators exploiting sparse regularization. We prove that many distributions revolving around maximum a posteriori (MAP) interpretation of sparse regularized estimators are in fact incompressible, in the limit of large problem sizes. We especially highlight the Laplace distribution and regularized estimators such as the Lasso and basis pursuit denoising. We rigorously disprove the myth that the success of minimization for compressed sensing image reconstruction is a simple corollary of a Laplace model of images combined with Bayesian MAP estimation, and show that in fact quite the reverse is true. To establish this result, we identify nontrivial undersampling regions where the simple least-squares solution almost surely outperforms an oracle sparse solution, when the data are generated from the Laplace distribution. We also provide simple rules of thumb to characterize classes of compressible and incompressible distributions based on their second and fourth moments. Generalized Gaussian and generalized Pareto distributions serve as running examples.