Estimation and testing in low rank multivariate models

在大数据时代，我们收集的观测数据涉及越来越多的变量和案例。 即使在数据的初步缩减之后，感兴趣的子集也可能具有高维度和可观的样本量。 这些数据中有趣的结构通常是低维的，实际上，这些模型出现在许多科学领域，从计量经济学到基因组学和信号处理等等。 本计画将研究一类特殊的低维结构，即比例单位矩阵或对角矩阵的低秩扰动的估计与检验。 它将考虑多维统计方法的高维版本，这些方法已广泛用于传统数据：主成分，多响应回归，典型相关等，以及矩阵去噪等较新的应用。该项目将研究变量数量和样本大小处于同一数量级的比例限制设置。 它将探讨相变现象的一个广泛的多元方法，部分使用系统的框架，由A。T. James. 下面的相变的邻接属性将被调查，将高斯行为以上的临界点。 将使用单独的低噪声近似值来推导最大根检验的长期寻求的功效近似值。 在估计中，该项目将研究对应于低秩结构的经验特征值的最佳收缩程序，明确结果如何强烈依赖于所选择的特定损失函数。 标量非线性以及阈值技术都将被考虑。 该项目将建立在协方差估计和矩阵去噪的初步工作的基础上，并为其他多变量设置，如低秩因子模型，典型相关和判别分析开发结果。

In the big data era, observations are collected on ever larger numbers of variables and cases. Even after preliminary reductions of the data, subsets of interest may have high dimensionality and appreciable sample size. The interesting structure in such data is often low dimensional, indeed such models occur in many scientific domains from econometrics to genomics and signal processing and well beyond. This project will investigate the estimation and testing of a particular class of low dimensional structures, namely low rank perturbations of scaled identity or diagonal matrices. It will consider high dimensional versions of multivariate statistical methods that have found wide use for traditional data: principal components, multiple response regression, canonical correlations etc., as well as newer applications such as matrix denoising.The project will study the proportional limit setting in which the number of variables and the sample size are of the same order of magnitude. It will explore the phase transition phenomenon for a wide class of multivariate methods, using in part the systematic framework developed by A. T. James. Contiguity properties below the phase transition will be investigated as will Gaussian behavior above the critical point. A separate low noise approximation will be used to derive long sought power approximations for largest root tests. In estimation, the project will study optimal shrinkage procedures for the empirical eigenvalues that correspond to the low rank structure, making explicit how the results depend strongly on the particular loss function chosen. Both scalar non-linearities as well as thresholding techniques will be considered. The project will build upon preliminary work for covariance estimation and matrix denoising, and also develop results for other multivariate settings such as low rank factor models, canonical correlations and discriminant analysis.