A general framework for Bayes structured linear models
A general framework for Bayes structured linear models
复制标题
DOI：
10.1214/19-aos1909
复制
发表时间：
2015-06
期刊：
影响因子：
--
通讯作者：
Chao Gao;A. Vaart;Harrison H. Zhou
中科院分区：
文献类型：
--
作者：
Chao Gao;A. Vaart;Harrison H. Zhou
高维统计处理从复杂的模型设置中提取结构化信息的挑战。与越来越多的频率论方法相比，很少有理论上最优的贝叶斯方法可以处理非常一般的高维模型。相比之下，贝叶斯方法在各种非参数设置中得到了广泛的研究，并且已经建立了速率最优后验收缩结果。本文提供了一个统一的方法来贝叶斯高维统计和贝叶斯非参数的一般框架结构线性模型。利用提出的两步模型选择先验，我们证明了一个抽象设置下的后验压缩的一般定理。主要定理可用于在许多复杂的模型设置下，包括随机块模型，图子估计和字典学习的最优后验收缩的新结果。它也可以用来重新获得最佳后收缩的问题，如稀疏线性回归和非参数聚集，这改善了以前的贝叶斯结果，这些问题。成功的关键在于提出了两步先验分布。参数的先验是椭圆拉普拉斯分布，其能够对具有大幅度的信号进行建模，并且模型的先验涉及补偿椭圆拉普拉斯分布的归一化常数的影响的重要校正因子。
High dimensional statistics deals with the challenge of extracting structured information from complex model settings. Compared with the growing number of frequentist methodologies, there are rather few theoretically optimal Bayes methods that can deal with very general high dimensional models. In contrast, Bayes methods have been extensively studied in various nonparametric settings and rate optimal posterior contraction results have been established. This paper provides a unified approach to both Bayes high dimensional statistics and Bayes nonparametrics in a general framework of structured linear models. With the proposed two-step model selection prior, we prove a general theorem of posterior contraction under an abstract setting. The main theorem can be used to derive new results on optimal posterior contraction under many complex model settings including stochastic block model, graphon estimation and dictionary learning. It can also be used to re-derive optimal posterior contraction for problems such as sparse linear regression and nonparametric aggregation, which improve upon previous Bayes results for these problems. The key of the success lies in the proposed two-step prior distribution. The prior on the parameters is an elliptical Laplace distribution that is capable to model signals with large magnitude, and the prior on the models involves an important correction factor that compensates the effect of the normalizing constant of the elliptical Laplace distribution.