On Statistical Modeling and Parameter Estimation for High Dimensional Systems

在过去的几十年里，数据收集和获取技术的巨大进步使科学家能够收集大量的高维数据，从而能够监测和研究复杂的系统。由于其固有的性质，许多高维数据集，如组学和全基因组关联研究(GWAS)数据，与维度相比，样本大小要小得多(称为小-n-大-P问题)。目前关于小n大P数据统计建模的研究主要集中在线性模型和广义线性模型上。然而，这些方法往往不适合于复杂系统的建模，并且模型参数的估计具有挑战性。该项目解决了两个基本问题，即统计建模和参数估计，以实现对高维数据的有效统计分析。该项目的成功完成将产生用于高维复杂系统统计推断的动手工具，这可以使许多科学和技术领域的研究人员受益。特别是，拟议的生物医学研究应用将带来准确的工具，用于检测与疾病过程相关的生物标记物，并为患有复杂疾病的个体患者量身定做最佳治疗方案。研究结果将通过合作、会议报告、书籍和将在学术期刊上发表的文章向统计和生物医学界传播。该项目还将通过让研究生参与该项目并将成果纳入本科生和研究生课程，对教育产生重大影响。此外，该项目下开发的R包将为高维数据的统计分析提供一个有价值的工具。现有的小n大P数据建模方法侧重于线性和广义线性模型，并通过对参数值施加稀疏约束来将问题转化为变量选择问题。虽然这些模型具有简单、计算效率高等优点，但参数估计仍然是一个具有挑战性的问题。虽然正则化在这些情况下经常使用，但当样本大小较小且变量高度相关时，正则化可能会表现不佳。为了解决这些问题，提出了两种新的方法，即贝叶斯神经网络(BNN)和块坐标一致性(BCC)。BNN方法首先用前馈神经网络对数据进行拟合，在贝叶斯框架下通过网络结构选择进行变量选择，然后通过并行计算解决相关的计算困难。与现有方法相比，对于高维非线性系统，BP神经网络可以更精确地选择相关变量和进行结果预测。BCC方法的工作原理是，使用循环算法最大化新的目标函数，即对数似然函数的期望，并根据其他参数的当前估计，迭代地为每个参数块找到一致的估计。BCC方法将高维参数估计问题归结为一系列低维参数估计问题。初步结果表明，与正则化方法相比，BCC在参数估计和变量选择方面都有明显的改进。所提出的方法的有效性将被严格研究并应用于生物标志物的发现、精确医学以及高维多元回归中回归系数和精度矩阵的联合估计。

The dramatic improvements in data collection and acquisition technologies over the last decades have enabled scientists to collect massive amounts of high-dimensional data that allow for monitoring and studying of complex systems. Due to their intrinsic nature, many of the high-dimensional datasets, such as omics and genome-wide association study (GWAS) data, have a much smaller sample size compared to the dimension (referred to as the small-n-large-P problem). Current research on statistical modeling of small-n-large-P data focuses on linear and generalized linear models. However, these approaches are often not adequate for modeling complex systems, and estimation of the model parameters is challenging. This project addresses two fundamental problems, statistical modeling and parameter estimation, toward a valid statistical analysis of high-dimensional data. Successful completion of this project will generate hands-on tools for statistical inference of high-dimensional complex systems, which can benefit researchers in many areas of science and technology. In particular, the proposed applications to biomedical studies will lead to accurate tools for detecting biomarkers associated with disease processes and tailoring optimal therapy for individual patients with complex diseases. The research results will be disseminated to the statistical and biomedical communities, via collaboration, conference presentations, books, and articles to be published in academic journals. The project will also have significant impact on education through the involvement of graduate students in the project, and incorporation of results into undergraduate and graduate courses. In addition, the R package developed under this project will provide a valuable tool for statistical analysis of high-dimensional data.The current approach to modeling small-n-large-P data focuses on linear and generalized linear models, and casts the problem as variable selection by imposing a sparsity constraint on parameter values. Although these models have many advantages, such as simplicity and computational efficiency, estimation of the parameters is still a challenging problem. While regularization is often used in these situations, it can perform poorly when the sample size is small and the variables are highly correlated. Two new methods are proposed to address these concerns, namely, Bayesian neural network (BNN) and blockwise coordinate consistency (BCC). The BNN method works by first fitting the data with a feed-forward neural network, conducting variable selection through network structure selection under a Bayesian framework, and resolving the associated computational difficulty via parallel computing. Compared to existing methods, BNN can lead to much more precise selection of relevant variables and outcome prediction for high-dimensional nonlinear systems. The BCC method works by maximizing a new objective function, the expectation of the log-likelihood function, using a cyclic algorithm and iteratively finding consistent estimates for each block of parameters conditional on the current estimates of the other parameters. The BCC method reduces the high-dimensional parameter estimation problem to a series of low-dimensional parameter estimation problems. The preliminary results indicate that BCC can provide a drastic improvement in both parameter estimation and variable selection over regularization methods. The validity of the proposed methods will be rigorously studied and applied to biomarker discovery, precision medicine, and joint estimation of the regression coefficients and precision matrix for high-dimensional multivariate regression.