Spectral and principal components analysis in sparse, high-dimensional data

随着数据收集技术的快速发展，关系数据在现代科学中变得越来越重要。一般而言，关系数据记录了感兴趣群体中参与者之间的交互和依赖关系。一个典型的例子是网络数据，例如社交网络、万维网和恐怖分子网络，其中每个参与者由网络中的节点表示，而两个参与者之间的交互由相应节点之间的边的存在来表示。关系数据的另一种常见形式是协方差和相关数据，它总结了行为者之间的成对依赖关系，例如基因-基因共表达、脑成像中的功能相关性以及大气和海洋测量中的空间相关性。这样的数据集通常包含重要的结构，可以为感兴趣的人群提供关键的见解。例如，网络数据中的参与者群体可以分为具有不同连通性模式的几个社区；相关性数据中的群体可能包含几个重要的参与者，这些参与者可以解释大多数观察到的可变性。然而，这些数据集的高维和复杂的依赖结构使得恢复这些隐藏结构成为一个具有挑战性的统计问题，本研究旨在提出利用谱分析和主成分分析对网络和协方差数据进行统计推断的理论和方法。这两个主题被结合在一起，并使用最近在随机矩阵理论、频谱分析和经验过程理论中开发的一组新工具进行研究。本项目将研究三个主题。第一个主题是使用谱聚类来更好地理解和改进稀疏网络模型中的社区恢复，谱聚类是文献和实践中最流行的方法之一。第二个主题是统计极小极大框架中的网络社区检测，包括由模型参数的综合集合量化的信息理论下界，以及达到下界的最优估计过程。第三个主题是一般稀疏主成分分析模型的拟合优度检验，其中将使用检测边界框架开发自适应程序；将通过考虑Sobolev椭球等规则替代方案来解决高维挑战。

With the rapid advances in data collection technology, relational data is becoming increasingly important in modern sciences. Broadly speaking, relational data records interactions and dependences among actors in a population of interest. A typical example is network data, such as social networks, world-wide-web, and terrorist networks, where each actor is represented by a node in the network and an interaction between two actors is represented by the presence of an edge between the corresponding nodes. Another common form of relational data is covariance and correlation data, which summarizes pairwise dependence among actors, such as gene-gene co-expression, functional correlation in brain imaging, and spatial correlation in atmospheric and oceanographic measurements. Such data sets often contain important structures that can provide key insights to the population of interest. For example, the population of actors in a network data may be divided into several communities with different connectivity patterns; the population in a correlation data may contain a few important actors that account for most of the observed variability. However, the high dimensionality and complex dependence structure in these data sets make it a challenging statistical problem to recover these hidden structures.This research project aims at advancing the theory and methodology in statistical inference for network and covariance data using spectral and principal components analysis. These two topics are brought together and studied using a novel set of tools recently developed in random matrix theory, spectral analysis, and empirical process theory. This project will investigate three topics. The first topic is a better understanding and refinement of community recovery in sparse network models using spectral clustering, one of the most popular methods in the literature and in practice. The second topic is network community detection in a statistical minimax framework, including information-theoretic lower bounds quantified by a comprehensive collection of model parameters, and optimal estimation procedures that achieve the lower bounds. The third topic is goodness-of-fit tests for general sparse principal components analysis models, where adaptive procedures will be developed using a detection boundary framework; and the high dimensionality challenge will be tackled by considering regular alternatives such as Sobolev ellipsoids.