Collaborative Research: Optimal Bayesian Concentration Rates from Double Empirical Priors

协作研究：来自双重经验先验的最佳贝叶斯浓度

• 批准号: 1506879
• 负责人: Stephen Walker
• 金额: $ 12.56万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-08-01 至 2018-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1506879&HistoricalAwards=false
• 关键词: Collaborative; Research; Optimal; Bayesian; Concentration

Statisticians frequently encounter problems that involve complicated models with high-dimensional parameters, particularly in "big data" settings. From a Bayesian perspective, it is imperative in these problems that the prior distribution be chosen to sit in a good position. Information about where is a good starting position can come from the data. There is a potential danger with this basic strategy, namely, that a double use of data might cause the model to track the data too closely, resulting on over fitting. To avoid this, the PIs introduce a regularization technique that suitably re-weights the likelihood, preventing the model from learning too quickly. This general "double empirical Bayes" strategy, where the prior is centered on the data and the likelihood is re-weighted, will be applied to several important and challenging high-dimensional problems, including estimation of sparse high-dimensional precision matrices, which is relevant to estimation of large complex networks. In this project, the PIs will develop this new double empirical Bayes framework for inference on high-dimensional parameters with a relatively low "complexity" or "effective dimension". For example, in function estimation problems, posited smoothness on the function is a constraint on its complexity. The first step of the double empirical Bayes strategy is to use a prior, indexed by the complexity of the parameter, centered at a complexity-specific estimate of the parameter based on data. To prevent the posterior from tracking the data too closely, the second step is to re-weight the likelihood to be combined with the data-dependent prior. The result is a sort of posterior distribution on the parameter space, and the PIs will provide general conditions for this posterior to concentrate around the truth at optimal rates. An additional advantage of this new approach is that the complexity-specific priors, for suitable centering, can be taken of relatively simple form, which facilitates computation. The PIs will investigate the double empirical Bayes analysis of several important high-dimensional inference problems, including density and function estimation, variable selection problems in non-linear models, and estimation of sparse precision matrices. Software will be developed for each application.

统计学家经常遇到涉及高维参数的复杂模型的问题，特别是在“大数据”设置中。从贝叶斯的角度来看，在这些问题中，先验分布必须选择在一个好的位置。关于哪里是一个好的开始位置的信息可以来自数据。这种基本策略存在一个潜在的危险，即数据的双重使用可能会导致模型过于紧密地跟踪数据，从而导致过度拟合。为了避免这种情况，PI引入了一种正则化技术，可以适当地重新加权可能性，防止模型学习得太快。这种一般的“双经验贝叶斯”策略，先验知识以数据为中心，似然性被重新加权，将被应用于几个重要的和具有挑战性的高维问题，包括稀疏高维精度矩阵的估计，这与大型复杂网络的估计有关。在这个项目中，PI将开发这种新的双经验贝叶斯框架，用于以相对较低的“复杂性”或“有效维度”对高维参数进行推断。例如，在函数估计问题中，函数上的假定平滑性是对其复杂性的约束。双经验贝叶斯策略的第一步是使用先验，由参数的复杂性索引，以基于数据的参数的复杂性特定估计为中心。为了防止后验过于接近数据，第二步是重新加权与数据相关先验相结合的可能性。结果是参数空间上的一种后验分布，PI将为该后验分布提供一般条件，以最佳速率集中在真相周围。这种新方法的另一个优点是，对于合适的中心化，特定于复杂性的先验可以采用相对简单的形式，这有利于计算。PI将研究几个重要的高维推理问题的双经验贝叶斯分析，包括密度和函数估计，非线性模型中的变量选择问题，以及稀疏精度矩阵的估计。将为每个应用程序开发软件。