On Statistical Modeling and Parameter Estimation for High Dimensional Systems

在过去的几十年里，数据收集和获取技术的巨大进步使科学家能够收集大量的高维数据，以便对复杂系统进行监测和研究。由于其固有的性质，许多高维数据集，如组学和全基因组关联研究（GWAS）数据，与维度（称为小n大p问题）相比，具有小得多的样本量。目前对小n大p数据统计建模的研究主要集中在线性和广义线性模型上。然而，这些方法往往不适合复杂系统的建模，并且模型参数的估计是具有挑战性的。该项目解决了两个基本问题，统计建模和参数估计，朝着有效的高维数据统计分析。该项目的成功完成将为高维复杂系统的统计推断提供实用工具，这将使许多科学和技术领域的研究人员受益。特别是，在生物医学研究中的应用将导致精确的工具来检测与疾病过程相关的生物标志物，并为患有复杂疾病的个体患者量身定制最佳治疗。研究结果将通过合作、会议发言、书籍和将在学术期刊上发表的文章传播给统计和生物医学界。该项目还将通过研究生参与该项目，并将结果纳入本科和研究生课程，对教育产生重大影响。此外，在这个项目下开发的R包将为高维数据的统计分析提供有价值的工具。目前对小n大p数据建模的方法侧重于线性和广义线性模型，并通过对参数值施加稀疏性约束来将问题作为变量选择。尽管这些模型具有简单和计算效率高的优点，但参数的估计仍然是一个具有挑战性的问题。虽然正则化经常用于这些情况，但当样本量很小且变量高度相关时，它可能会表现不佳。为了解决这些问题，提出了两种新的方法，即贝叶斯神经网络（BNN）和块坐标一致性（BCC）。BNN方法首先用前馈神经网络拟合数据，在贝叶斯框架下通过网络结构选择进行变量选择，并通过并行计算解决相关的计算难度。与现有方法相比，BNN可以更精确地选择相关变量和预测高维非线性系统的结果。BCC方法的工作原理是最大化一个新的目标函数，即对数似然函数的期望，使用循环算法，并以其他参数的当前估计为条件，迭代地找到每个参数块的一致估计。BCC方法将高维参数估计问题简化为一系列低维参数估计问题。初步结果表明，与正则化方法相比，BCC在参数估计和变量选择方面都有显著改善。该方法的有效性将被严格研究并应用于生物标志物发现、精准医学、高维多元回归回归系数和精度矩阵的联合估计。

The dramatic improvements in data collection and acquisition technologies over the last decades have enabled scientists to collect massive amounts of high-dimensional data that allow for monitoring and studying of complex systems. Due to their intrinsic nature, many of the high-dimensional datasets, such as omics and genome-wide association study (GWAS) data, have a much smaller sample size compared to the dimension (referred to as the small-n-large-P problem). Current research on statistical modeling of small-n-large-P data focuses on linear and generalized linear models. However, these approaches are often not adequate for modeling complex systems, and estimation of the model parameters is challenging. This project addresses two fundamental problems, statistical modeling and parameter estimation, toward a valid statistical analysis of high-dimensional data. Successful completion of this project will generate hands-on tools for statistical inference of high-dimensional complex systems, which can benefit researchers in many areas of science and technology. In particular, the proposed applications to biomedical studies will lead to accurate tools for detecting biomarkers associated with disease processes and tailoring optimal therapy for individual patients with complex diseases. The research results will be disseminated to the statistical and biomedical communities, via collaboration, conference presentations, books, and articles to be published in academic journals. The project will also have significant impact on education through the involvement of graduate students in the project, and incorporation of results into undergraduate and graduate courses. In addition, the R package developed under this project will provide a valuable tool for statistical analysis of high-dimensional data.The current approach to modeling small-n-large-P data focuses on linear and generalized linear models, and casts the problem as variable selection by imposing a sparsity constraint on parameter values. Although these models have many advantages, such as simplicity and computational efficiency, estimation of the parameters is still a challenging problem. While regularization is often used in these situations, it can perform poorly when the sample size is small and the variables are highly correlated. Two new methods are proposed to address these concerns, namely, Bayesian neural network (BNN) and blockwise coordinate consistency (BCC). The BNN method works by first fitting the data with a feed-forward neural network, conducting variable selection through network structure selection under a Bayesian framework, and resolving the associated computational difficulty via parallel computing. Compared to existing methods, BNN can lead to much more precise selection of relevant variables and outcome prediction for high-dimensional nonlinear systems. The BCC method works by maximizing a new objective function, the expectation of the log-likelihood function, using a cyclic algorithm and iteratively finding consistent estimates for each block of parameters conditional on the current estimates of the other parameters. The BCC method reduces the high-dimensional parameter estimation problem to a series of low-dimensional parameter estimation problems. The preliminary results indicate that BCC can provide a drastic improvement in both parameter estimation and variable selection over regularization methods. The validity of the proposed methods will be rigorously studied and applied to biomarker discovery, precision medicine, and joint estimation of the regression coefficients and precision matrix for high-dimensional multivariate regression.