Development of Statistical Methods for High-dimensional and Complex Data

随着技术的进步，科学家面临着越来越多的高维复杂数据的挑战。例如，来自微阵列实验的遗传数据非常大，需要新技术来识别各种疾病的特定基因。随着时间的推移，数百万人的各种变量的纵向数据产生了有趣的挑战。这些数据需要新的统计技术。高维数据中的一些挑战包括从一大组变量中选择变量。一些现有的方法也遭受高的错误发现率。此外，在分位数回归方法中，需要解决分位数交叉的奇怪现象。最后，空间和纵向研究需要特殊的有效方法来估计协方差模式。由于高维或复杂数据的不同特征，PI开发了几种方法。在这份补助金申请中，PI建议：1。用于高维数据的变量选择的新技术和降低错误发现率的新方法; 2.处理分位数交叉现象的新技术及其在类概率估计中的应用;空间数据和纵向数据协方差结构估计的新方法; 4.拟似然方法的参数引导非参数估计。高维变量选择技术被许多科学家要求有效地分析大规模复杂的金融、环境和生物医学数据，如基因表达、蛋白质组学和代谢组学，或脑成像数据。这些类型的数据需要识别重要特征的技术。为了实现这一目标，PI提出了一种筛选方法来选择适当的统计模型。这种筛选方法可以应用于生物医学数据，以定位负责感兴趣的疾病，如乳腺癌和白血病的重要基因。在环境和临床研究中，空间和纵向数据也是稀疏和不规则的。对于这类数据，人们一直致力于研究它们的协方差结构。在这个提议中，PI提出了一种灵活的基于卷积的方法来非参数地估计协方差结构。这种方法可以应用于许多环境数据，如降水和风，以提高我们对环境变化的理解，包括众所周知的“气候变化”问题。这项研究有许多社会应用。此外，PI利用该部门的辅导计划与美国博士生，特别是妇女和少数民族合作。PI还与NSF-CSUMS项目的本科生合作，因为培养未来计算能力强的批判性思维者非常重要。

As technology advances, scientists are challenged by more and more high-dimensional and complex data. For example, genetic data from microarray experiments are very large in size and new techniques are needed to identify specific genes for various diseases. Longitudinal data on various variables over time on millions of individuals produce interesting challenges. Such data call for new statistical techniques. Some of the challenges in high-dimensional data include variable selection from a large group of variables. Some of the existing methods also suffer from a high false discovery rate. In addition, in quantile regression methods, an odd phenomenon of quantile crossing needs to be addressed. Finally, spatial and longitudinal studies require special efficient methods for estimating the covariance patterns. Motivated by different features of high-dimensional or complex data, the PI develops several methods. In this grant application, the PI proposes: 1. new techniques for variable selection for high-dimensional data and new methods to reduce the false discovery rate; 2. new techniques to handle the phenomenon of quantile crossing with application to class probability estimation; 3. new methods to estimate covariance structure for spatial data and longitudinal data; and 4. parametrically guided nonparametric estimation for the quasi-likelihood method. The proposed methods will be studied theoretically for their asymptotic behavior and compared with some of the existing methods both theoretically and through simulations.High-dimensional variable selection techniques are called for by many scientists to efficiently analyze large-scale complex financial, environmental, and biomedical data such as gene expression, proteomics and metabolomics, or brain imaging data. These types of data require techniques to identify important features. To achieve this goal, the PI proposes a screening method to select appropriate statistical models. This screening method can be applied to biomedical data to locate important genes responsible for diseases of interest such as breast cancer and leukemia. Spatial and longitudinal data are sparsely and irregularly observed also in environmental and clinical studies. For such data, many efforts have been devoted to studying their covariance structures. In this proposal, the PI proposes a flexible convolution-based method to estimate covariance structures nonparametrically. This method can be applied to many environmental data such as precipitation and wind to improve our understanding of environmental changes including the well-known "climate change" issue. This research has many societal applications. In addition, the PI takes advantage of the mentoring program in the department to work with US doctoral students, especially women and minorities. The PI also works with undergraduates from NSF-CSUMS program in the department as it is important to train computationally strong critical thinkers for the future.