Estimation of High Dimensional Matrices of Low Effective Rank with Applications to Structural Copula Models

本建议的中心目标是：(a)在各种矩阵范数中，对有效秩降的高维协方差矩阵的样本协方差估计的准确性提供明确的有限样本界；(b)将这些结果推广到功能数据，并描述有效秩降低的协方差算子的类别。利用这些结果开发完全数据驱动的方法，具有强大的理论依据，在有限样本中选择特征值和特征向量。将这些结果应用于汽车尾气排放模型；(c)研究椭圆轴高维相关矩阵的因子模型。获得这些矩阵的极大极小估计量，并将这些结果用于乳腺癌数据的分类问题。我们提出的研究与现有的关于稀疏性约束下协方差或相关矩阵估计的结果之间存在有趣的联系。然而，现有稀疏性类型（入口型、行型、非对角线衰减）下的估计不能用于一般依赖类型的建模。提出的工作弥补了这一差距，并提出了不同的数学和计算挑战。对高维数据进行建模并评估其变异性在许多科学学科中提出了越来越大的挑战。例如，这些挑战发生在遗传学和分子生物学的网络数据建模中；经济学中的高维投资组合；还有心理学、公共卫生、交通和城市规划方面的曲线样本。当数据由具有低维结构的模型生成时，基本上可以提供更好的解决方案。在高维协方差和相关矩阵估计的统计问题中，本提案将制定低维结构的相关概念（如低有效秩或近似低维因子模型）。对高维模型中各类协方差矩阵进行系统研究的必要性，特别是在函数数据设置中，直到最近几年才开始认识到。因此，这一建议是对这一重要领域目前有限的方法和理论结果的及时补充。这些技术的实用性将通过对基因组学、蛋白质组学和环境工程数据的应用来证明。实现所开发方法的自由软件将以易于实现的形式在网络上提供。

The central goals of this proposal are:(a) To provide sharp finite sample bounds, in various matrix norms, on the accuracy of the sample covariance estimator of high dimensional covariance matrices of reduced effective rank;(b) To extend these results to functional data and characterize classes of covariance operators of reduced effective rank. To use these results to develop fully data driven methods, with strong theoretical justification, for eigenvalue and eigenvector selection, in finite samples. To apply these results to modeling vehicle emissions exhaust; (c) To study factor models of high dimensional correlation matrices of elliptical copulas. To obtain minimax estimators of these matrices and to use these results in classification problems in breast cancer data. There are interesting connections between our proposed research and existing results on estimation of covariance or correlation matrices under sparsity constrains. However, estimation under the existing sparsity types (entry-wise, row-wise, off-diagonal decay) cannot be used for modeling general types of dependency. The proposed work bridges this gap, and poses different mathematical and computational challenges. Modeling high dimensional data and evaluating their variability presents increasing challenges in many scientific disciplines. For instance, such challenges occur in modeling network data in genetics and molecular biology; high dimensional portfolios in economics; and samples of curves in psychology, public health, transportation and urban planning. Substantially better solutions can be provided whenever the data is generated by a model with low dimensional structure. In the statistical problem of high dimensional covariance and correlation matrix estimation, this proposal will formulate the relevant notion of low dimensional structure (for instance, low effective rank or approximate low dimensional factor models). The need for a systematic investigation of various classes of covariance matrices in high dimensional models, especially in functional data settings, only begun to be recognized in recent years. This proposal is therefore a timely addition to the currently limited battery of methods and theoretical results in this important area. The usefulness of these techniques will be demonstrated by applications to data from genomics, proteomics and environmental engineering. Free software that implements the developed methodology will be made available on the web in a readily implementable form.