Spectral and principal components analysis in sparse, high-dimensional data

随着数据采集技术的飞速发展，关系数据在现代科学中的地位日益重要。广义地说，关系数据记录了感兴趣群体中参与者之间的交互和依赖关系。一个典型的例子是网络数据，如社交网络、万维网和恐怖网络，其中每个参与者由网络中的一个节点表示，两个参与者之间的交互由相应节点之间的边缘表示。另一种常见的关联数据形式是协方差和相关数据，它总结了参与者之间的两两依赖关系，如基因-基因共表达、脑成像中的功能相关性、大气和海洋测量中的空间相关性。这些数据集通常包含重要的结构，可以为感兴趣的总体提供关键的见解。例如，网络数据中的参与者可能被划分为具有不同连接模式的几个社区；相关数据中的总体可能包含几个重要的因素，这些因素可以解释大部分观察到的变异性。然而，这些数据集的高维和复杂的依赖结构使得这些隐藏结构的恢复成为一个具有挑战性的统计问题。本研究项目旨在利用谱分析和主成分分析，推进网络和协方差数据统计推断的理论和方法。这两个主题结合在一起，并使用随机矩阵理论、谱分析和经验过程理论中最近开发的一套新工具进行研究。这个项目将调查三个主题。第一个主题是使用谱聚类更好地理解和改进稀疏网络模型中的群落恢复，谱聚类是文献和实践中最流行的方法之一。第二个主题是统计极大极小框架下的网络社区检测，包括由模型参数的综合集合量化的信息论下界，以及实现下界的最优估计程序。第三个主题是一般稀疏主成分分析模型的拟合优度检验，其中将使用检测边界框架开发自适应程序；而高维的挑战将通过考虑常规的替代方案来解决，比如索博列夫椭球。

With the rapid advances in data collection technology, relational data is becoming increasingly important in modern sciences. Broadly speaking, relational data records interactions and dependences among actors in a population of interest. A typical example is network data, such as social networks, world-wide-web, and terrorist networks, where each actor is represented by a node in the network and an interaction between two actors is represented by the presence of an edge between the corresponding nodes. Another common form of relational data is covariance and correlation data, which summarizes pairwise dependence among actors, such as gene-gene co-expression, functional correlation in brain imaging, and spatial correlation in atmospheric and oceanographic measurements. Such data sets often contain important structures that can provide key insights to the population of interest. For example, the population of actors in a network data may be divided into several communities with different connectivity patterns; the population in a correlation data may contain a few important actors that account for most of the observed variability. However, the high dimensionality and complex dependence structure in these data sets make it a challenging statistical problem to recover these hidden structures.This research project aims at advancing the theory and methodology in statistical inference for network and covariance data using spectral and principal components analysis. These two topics are brought together and studied using a novel set of tools recently developed in random matrix theory, spectral analysis, and empirical process theory. This project will investigate three topics. The first topic is a better understanding and refinement of community recovery in sparse network models using spectral clustering, one of the most popular methods in the literature and in practice. The second topic is network community detection in a statistical minimax framework, including information-theoretic lower bounds quantified by a comprehensive collection of model parameters, and optimal estimation procedures that achieve the lower bounds. The third topic is goodness-of-fit tests for general sparse principal components analysis models, where adaptive procedures will be developed using a detection boundary framework; and the high dimensionality challenge will be tackled by considering regular alternatives such as Sobolev ellipsoids.