Collaborative Research: Bayesian and Semi-Bayesian Methods for Detecting Relationships in High Dimensions

在这个大数据时代，海量数据集正在被例行公事地生成，我们看到越来越多的人需要强大、可靠和可解释的统计学习工具来帮助理解这些数据。这个项目的主要思想和方法侧重于开发有效的统计学习工具，以了解复杂和不同的结构，例如那些随时间变化的结构或在不同个人群体中变化的高维度结构。这些活动将对高维贝叶斯分析和非线性关系建模产生重大影响。虽然目前大多数高维贝叶斯分析的努力都集中在线性模型上，但这个项目专注于两种推广标准线性模型的方法来满足某些实际挑战：一种是混合建模的广义形式，称为个性化变量选择，它使每个个体观测通过使用神经元先验来拥有自己的因变量集。另一个扩展是形成混合结构的指数模型的贝叶斯推断。该项目将产生有用的工具(或定制软件)，用于在大量潜在变量中发现可解释的非线性和交互模式。我们算法中集成的统计建模、设计和学习策略的各个方面广泛适用于涉及复杂系统和高维数据中的信号发现的问题。该项目还将为研究生提供教育和跨学科研究机会，并将产生对生物医学研究人员、经济学家、社会科学家和许多其他从业者有用的软件。在大量的回归问题中，特别是在高维环境下，手头的协变量与目标兴趣量之间的关联结构可能在观测上是异质的，这就需要有效的方法来检测这种非平凡结构。标准程序，包括传统的变量选择，通常忽略了这些不同因素之间相互作用的存在。这一研究项目旨在开发统计程序，以灵活和计算高效的方式确定回答Y和一组协变量X之间的复杂关系。项目1的重点是贝叶斯个体化变量选择(BIVS)，它推广了标准的线性回归模型，以量化因变量不同且大小不同的个体观测之间的异质性影响。PI将研究其理论属性，包括模型选择的一致性和当模型假设被违反时的稳健性。项目2致力于开发一种有效的贝叶斯方法，通过一般指数模型来推断响应和协变量之间的半参数关系。PI将探索其计算可行性和理论性质，如在估计足够的降维空间上的后验收缩速度。项目3的重点是通过神经网络采用产生式过程来快速调整参数选择过程。通过使用该程序，可以有效地对一般模型进行交叉验证，如BIVS和贝叶斯指数模型、正则化变量选择和非参数函数估计。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

In this big-data era, massive data sets are being generated routinely and we are seeing a growing need for powerful, reliable, and interpretable statistical learning tools to help understand these data. The main ideas and approaches in this projectl focus on developing effective statistical learning tools to learn about complex and heterogeneous structures, such as those changing in time or varying among different groups of individuals, in high-dimensions. The activities will have a significant impact on high dimensional Bayesian analysis and modeling of nonlinear relationships. While most current efforts for high-dimensional Bayesian analyses have been focused on linear models, this project focuses on two ways of generalizing standard linear models to meet certain practical challenges: one is a generalized form of mixture modeling, termed as individualized variable selection, which enables each individual observation to have its own set of dependent variables through the employment of neuronized priors. Another extension is the Bayesian inference of index models that form a mixture structure. The project will lead to useful tools (or customized software) for discovering interpretable nonlinear and interactive patterns among a large number of potential variables. Various aspects of statistical modeling, design, and learning strategies integrated in our algorithms are broadly applicable to problems involving signal discovery in complex systems and high-dimensional data. The project will also provide both educational and interdisciplinary research opportunities for graduate students, and will result in software useful to biomedical researchers, economists, social scientists, and many other practitioners. In a vast number of regression problems, especially under high-dimensional settings, the structure of the association between covariates in hand and the target quantity of interest might be heterogeneous over observations, which calls for effective methods to detect such non-trivial structures. Standard procedures, including traditional variable selections, commonly overlook the existence of interplays of these heterogeneous factors. This research project aims to develop statistical procedures that identify the complicated relationship between response Y and a set of covariates X in flexible and computationally efficient ways. Project 1 focuses on Bayesian individualized variable selection (BIVS), which generalizes standard linear regression models to quantify heterogeneous effects among individual observations that differ in their dependent variables with different magnitudes. The PIs will investigate its theoretical properties, including model selection consistency and its robustness when the model assumption is violated. Project 2 is devoted to the development of an efficient Bayesian method to infer the semi-parametric relationship between the response and covariates through general index models. The PIs will explore its computational feasibility and theoretical properties such as the posterior contraction rate on the estimation of the sufficient dimension reduction space. Project 3 focuses on a fast tuning parameter selection procedure by employing a generative process via neural networks. By using this procedure, the cross-validation can be efficiently implemented for general models, such as the BIVS and Bayesian index models, regularized variable selection, and nonparametric function estimation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.