Collaborative Research: Optimal Bayesian Concentration Rates from Double Empirical Priors

统计学家经常遇到涉及具有高维参数的复杂模型的问题，特别是在“大数据”环境中。从贝叶斯的角度来看，在这些问题中，必须选择先验分布，使其处于良好的位置。关于哪里是一个好的起点的信息可以从数据中获得。这一基本策略存在一个潜在的危险，即重复使用数据可能会导致模型跟踪数据过于紧密，从而导致过度拟合。为了避免这种情况，PI引入了一种正则化技术，该技术适当地重新加权可能性，防止模型学习过快。这种一般的“双经验贝叶斯”策略，其中先验以数据为中心，对似然进行重新加权，将被应用于几个重要和具有挑战性的高维问题，包括与大型复杂网络估计相关的稀疏高维精度矩阵的估计。在这个项目中，PIS将开发这个新的双经验贝叶斯框架，用于在相对较低的“复杂性”或“有效维度”的高维参数上进行推断。例如，在函数估计问题中，假定函数的光滑性是对其复杂性的限制。双重经验贝叶斯策略的第一步是使用以参数的复杂性为索引的先验，以基于数据的特定于复杂性的参数估计为中心。为了防止后验跟踪数据过于紧密，第二步是重新加权要与依赖于数据的先验相结合的可能性。结果是参数空间上的一种后验分布，PI将为这种后验以最优速率集中在真理周围提供一般条件。这种新方法的另一个优点是，对于适当的居中，特定于复杂性的先验可以采用相对简单的形式，这便于计算。PI将研究几个重要的高维推理问题的双重经验贝叶斯分析，包括密度和函数估计，非线性模型中的变量选择问题，以及稀疏精度矩阵的估计。将为每个应用程序开发软件。

Statisticians frequently encounter problems that involve complicated models with high-dimensional parameters, particularly in "big data" settings. From a Bayesian perspective, it is imperative in these problems that the prior distribution be chosen to sit in a good position. Information about where is a good starting position can come from the data. There is a potential danger with this basic strategy, namely, that a double use of data might cause the model to track the data too closely, resulting on over fitting. To avoid this, the PIs introduce a regularization technique that suitably re-weights the likelihood, preventing the model from learning too quickly. This general "double empirical Bayes" strategy, where the prior is centered on the data and the likelihood is re-weighted, will be applied to several important and challenging high-dimensional problems, including estimation of sparse high-dimensional precision matrices, which is relevant to estimation of large complex networks. In this project, the PIs will develop this new double empirical Bayes framework for inference on high-dimensional parameters with a relatively low "complexity" or "effective dimension". For example, in function estimation problems, posited smoothness on the function is a constraint on its complexity. The first step of the double empirical Bayes strategy is to use a prior, indexed by the complexity of the parameter, centered at a complexity-specific estimate of the parameter based on data. To prevent the posterior from tracking the data too closely, the second step is to re-weight the likelihood to be combined with the data-dependent prior. The result is a sort of posterior distribution on the parameter space, and the PIs will provide general conditions for this posterior to concentrate around the truth at optimal rates. An additional advantage of this new approach is that the complexity-specific priors, for suitable centering, can be taken of relatively simple form, which facilitates computation. The PIs will investigate the double empirical Bayes analysis of several important high-dimensional inference problems, including density and function estimation, variable selection problems in non-linear models, and estimation of sparse precision matrices. Software will be developed for each application.