High-Dimensional Inference beyond Linear Models

回归模型被广泛用于研究一组预测变量（所谓的协变量）和一些结果变量之间的关联。对回归系数及其置信区间的估计提供了有用的信息，例如，某些基因变异对肺癌的重要性，或与老年人口中记忆丧失相关的大脑区域。随着大数据时代的到来，具有多协变量的回归模型已被广泛用于解决基因组学、神经影像学、商业、工程、信息技术和其他生物医学研究等领域的许多重要科学问题，有时协变量的数量（如遗传变异）甚至大于样本量（如研究参与者的数量）。对大量协变量进行统计推断（即构建回归系数的置信区间）成为一个具有挑战性的问题，因为传统的方法如最大似然估计可能不存在或产生偏估计。近年来的研究表明，回归系数可以用正则化方法来估计，例如lasso方法。然而，众所周知，正则化方法会产生有偏差的估计，因此不能直接用于统计推断，特别是用于构建置信区间。一些研究人员已经证明，在实施巧妙的去偏过程后，可以在线性回归模型中做出适当的统计推断。然而，也发现去偏方法不适用于超出线性模型的情况。在不强加限制性假设的情况下，将为具有大量协变量的广义线性模型和Cox回归模型以及在脑成像研究中应用的功能回归模型开发理论和方法。将提供适当的分布理论和置信区间，从而使科学研究的结果更加可靠。现有的去偏方法在非线性模型（如广义线性模型或Cox模型）中不能成功地校正偏倚，导致统计推断结果较差。问题的主要原因包括对逆期望Hessian矩阵的不切实际的稀疏性假设，以及现有去偏方法中的“可忽略”项实际上是不可忽略的。在本项目中，将考虑两种方法来进一步消除lasso估计量的偏差，而不依赖于稀疏逆期望Hessian矩阵的假设：(i)当回归参数的数量小于样本量时，直接反演Hessian矩阵；（ii）在不使用Hessian矩阵逆的情况下消除主要偏差项——一种二次规划方法，它可以潜在地处理回归参数数量大于观测数量的情况。在具有高维协变量的Cox回归中出现了额外的挑战，其中所有观测值的基于部分似然的损失函数不是i.i.d，并且每个损失函数不是Lipschitz。所提出的方法将近似损失函数以产生i.i.d损失，并扩展到处理具有更复杂损失函数的多变量和聚类生存数据。针对脑成像数据，研究了基于Haar小波基的功能回归模型。增加的主要挑战是利用Haar小波来表征近似误差对精细去偏函数估计的渐近分布的影响。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Regression models are widely used in investigating the associations between a set of predicting variables, the so-called covariates, and some outcome variable. Estimates of regression coefficients and their confidence intervals provide useful information, for example, the importance of certain genetic variants to lung cancer, or brain regions associated with memory loss in an aging population. With the advent of big data era, regression models with many covariates have been commonly used to tackle many important scientific problems in areas such as genomics, neuroimaging, business, engineering, information technology, and other biomedical studies, and sometimes the number of covariates (e.g. genetic variants) is even greater than the sample size (e.g. the number of study participants). Making statistical inference (i.e. constructing confidence intervals for regression coefficients) for a large number of covariates becomes a challenging issue because the conventional methods such as the maximum likelihood estimation may either not exist or yield biased estimates. It has been shown in recent years that the regression coefficients can be estimated by using regularized methods, e.g., the lasso approach. However, it is also well-known that the regularized methods yield biased estimates, thus cannot be directly used for making statistical inference, in particular, for constructing confidence intervals. Some researchers have shown that proper statistical inference can be made in linear regression models after implementing a clever de-biasing procedure. However, it is also found that the de-biased method does not work beyond linear models. Without imposing restrictive assumptions, theory and methods will be developed for the generalized linear models and the Cox regression model with a large number of covariates, as well as for the functional regression models with applications in brain imaging studies. Proper distributional theory and confidence intervals will be provided, which will lead to more reliable results in scientific research. The existing de-biased methods do not successfully correct the bias in nonlinear models, e.g., the generalized linear models or the Cox model, leading to poor results in statistical inference. The main causes of the problem include the unrealistic sparsity assumption imposed on the inverse expected Hessian matrix, and that the "negligible" terms in the existing de-biased methods are in fact not negligible. In this project, two methods that further de-bias the lasso estimators without relying on the assumption of sparse inverse expected Hessian matrix will be considered: (i) directly inverting the Hessian matrix when the number of regression parameters is less than the sample size; (ii) eliminating the major bias term without using the inverse of Hessian matrix - a quadratic programming approach, which can potentially handle the case with larger number of regression parameters than the number of observations. Additional challenges arise in the Cox regression with high-dimensional covariates, where the partial-likelihood-based loss functions for all the observations are not i.i.d., and each loss function is not Lipschitz. The proposed method will be approximating the loss function to yield i.i.d. losses and extended to handling multivariate and clustered survival data with even more complicated loss functions. For the brain imaging data, functional regression model using Haar wavelet basis is investigated. The major added challenge is to characterize the impact of the approximation error using Haar wavelets on the asymptotic distribution of the refined de-biased functional estimation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.