Estimation and testing in low rank multivariate models

在大数据时代，观察数据收集的变量和案例数量越来越多。即使在初步减少数据之后，感兴趣的子集也可能具有高维度和可观的样本量。这些数据中有趣的结构通常是低维的，事实上，这些模型出现在许多科学领域，从计量经济学到基因组学和信号处理等等。该项目将研究一类特殊的低维结构的估计和测试，即缩放单位矩阵或对角矩阵的低秩摄动。它将考虑在传统数据中广泛使用的多元统计方法的高维版本：主成分、多响应回归、典型相关等，以及矩阵去噪等新应用。本项目将研究变量数与样本量处于同一数量级的比例极限设置。它将探索相变现象的一个广泛类别的多元方法，部分使用由a . T.詹姆斯开发的系统框架。将研究相变以下的连续性质以及临界点以上的高斯行为。一个单独的低噪声近似值将被用来推导长时间寻找的最大根测试的功率近似值。在估计中，该项目将研究与低秩结构相对应的经验特征值的最佳收缩程序，明确结果如何强烈依赖于所选择的特定损失函数。标量非线性和阈值技术都将被考虑。该项目将建立在协方差估计和矩阵去噪的初步工作基础上，并为其他多变量设置（如低秩因子模型、典型相关和判别分析）开发结果。

In the big data era, observations are collected on ever larger numbers of variables and cases. Even after preliminary reductions of the data, subsets of interest may have high dimensionality and appreciable sample size. The interesting structure in such data is often low dimensional, indeed such models occur in many scientific domains from econometrics to genomics and signal processing and well beyond. This project will investigate the estimation and testing of a particular class of low dimensional structures, namely low rank perturbations of scaled identity or diagonal matrices. It will consider high dimensional versions of multivariate statistical methods that have found wide use for traditional data: principal components, multiple response regression, canonical correlations etc., as well as newer applications such as matrix denoising.The project will study the proportional limit setting in which the number of variables and the sample size are of the same order of magnitude. It will explore the phase transition phenomenon for a wide class of multivariate methods, using in part the systematic framework developed by A. T. James. Contiguity properties below the phase transition will be investigated as will Gaussian behavior above the critical point. A separate low noise approximation will be used to derive long sought power approximations for largest root tests. In estimation, the project will study optimal shrinkage procedures for the empirical eigenvalues that correspond to the low rank structure, making explicit how the results depend strongly on the particular loss function chosen. Both scalar non-linearities as well as thresholding techniques will be considered. The project will build upon preliminary work for covariance estimation and matrix denoising, and also develop results for other multivariate settings such as low rank factor models, canonical correlations and discriminant analysis.