An equivalence between high dimensional Bayes optimal inference and M-estimation
An equivalence between high dimensional Bayes optimal inference and M-estimation
复制标题
高维贝叶斯最优推理与 M 估计之间的等价
DOI：
--
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
S. Ganguli
中科院分区：
文献类型：
--
作者：
Madhu S. Advani;S. Ganguli
当从噪声测量中恢复未知信号时，执行最优贝叶斯MMSE（最小均方误差）推断的计算困难通常需要使用最大后验（MAP）推断，这是正则化M估计的一种特殊情况，作为替代。然而，MAP是次优的高维，当未知信号分量的数量是类似的测量的数量。在这项工作中，我们证明，当信号分布和似然函数与噪声都是对数凹，最佳MMSE性能是渐近实现通过另一个M-估计过程。这个过程涉及最小化凸损失和正则化函数，这些函数是广泛应用的MAP优化问题的非线性平滑版本。我们的研究结果提供了一个新的启发式推导和解释最近发现的最佳M-估计在设置的线性测量和加性噪声，并进一步扩展这些结果的非线性测量与非加性噪声。我们数值上证明了我们的最佳M-估计相对于MAP的上级性能。总体而言，在我们的工作的核心是一个显着的等价性之间的启示两个看似非常不同的计算问题：即高维贝叶斯集成的MMSE推理，和高维凸优化的M-估计。在本质上，我们表明，前困难的积分可以通过解决后者，更简单的优化问题计算。
When recovering an unknown signal from noisy measurements, the computational difficulty of performing optimal Bayesian MMSE (minimum mean squared error) inference often necessitates the use of maximum a posteriori (MAP) inference, a special case of regularized M-estimation, as a surrogate. However, MAP is suboptimal in high dimensions, when the number of unknown signal components is similar to the number of measurements. In this work we demonstrate, when the signal distribution and the likelihood function associated with the noise are both log-concave, that optimal MMSE performance is asymptotically achievable via another M-estimation procedure. This procedure involves minimizing convex loss and regularizer functions that are nonlinearly smoothed versions of the widely applied MAP optimization problem. Our findings provide a new heuristic derivation and interpretation for recent optimal M-estimators found in the setting of linear measurements and additive noise, and further extend these results to nonlinear measurements with non-additive noise. We numerically demonstrate superior performance of our optimal M-estimators relative to MAP. Overall, at the heart of our work is the revelation of a remarkable equivalence between two seemingly very different computational problems: namely that of high dimensional Bayesian integration underlying MMSE inference, and high dimensional convex optimization underlying M-estimation. In essence we show that the former difficult integral may be computed by solving the latter, simpler optimization problem.