Spectral and principal components analysis in sparse, high-dimensional data

随着数据收集技术的快速发展，关系数据在现代科学中变得越来越重要。一般来说，关系数据记录了感兴趣的群体中的参与者之间的交互和依赖关系。典型的示例是网络数据，诸如社交网络、万维网和恐怖分子网络，其中每个参与者由网络中的节点表示，并且两个参与者之间的交互由对应节点之间的边的存在表示。 关系数据的另一种常见形式是协方差和相关性数据，它总结了参与者之间的成对依赖关系，例如基因-基因共表达，脑成像中的功能相关性以及大气和海洋测量中的空间相关性。 这些数据集通常包含重要的结构，可以为感兴趣的人群提供关键的见解。 例如，网络数据中的参与者群体可以被划分为具有不同连接模式的几个社区;相关性数据中的群体可能包含一些重要的参与者，这些参与者占大多数观察到的可变性。 然而，这些数据集中的高维数和复杂的相关结构使得恢复这些隐藏的结构成为一个具有挑战性的统计问题，本研究旨在利用谱分析和主成分分析来提高网络和协方差数据的统计推断的理论和方法。这两个主题汇集在一起，并使用一套新的工具，最近开发的随机矩阵理论，频谱分析和经验过程理论的研究。 本项目将研究三个主题。第一个主题是更好地理解和细化稀疏网络模型中使用谱聚类，在文献和实践中最流行的方法之一的社区恢复。第二个主题是网络社区检测的统计极小极大框架，包括信息理论的下限量化的一个全面的收集模型参数，最佳估计程序，实现了下限。第三个主题是一般稀疏主成分分析模型的拟合优度测试，其中自适应程序将使用检测边界框架开发;高维挑战将通过考虑Sobolev椭球等常规替代品来解决。

With the rapid advances in data collection technology, relational data is becoming increasingly important in modern sciences. Broadly speaking, relational data records interactions and dependences among actors in a population of interest. A typical example is network data, such as social networks, world-wide-web, and terrorist networks, where each actor is represented by a node in the network and an interaction between two actors is represented by the presence of an edge between the corresponding nodes. Another common form of relational data is covariance and correlation data, which summarizes pairwise dependence among actors, such as gene-gene co-expression, functional correlation in brain imaging, and spatial correlation in atmospheric and oceanographic measurements. Such data sets often contain important structures that can provide key insights to the population of interest. For example, the population of actors in a network data may be divided into several communities with different connectivity patterns; the population in a correlation data may contain a few important actors that account for most of the observed variability. However, the high dimensionality and complex dependence structure in these data sets make it a challenging statistical problem to recover these hidden structures.This research project aims at advancing the theory and methodology in statistical inference for network and covariance data using spectral and principal components analysis. These two topics are brought together and studied using a novel set of tools recently developed in random matrix theory, spectral analysis, and empirical process theory. This project will investigate three topics. The first topic is a better understanding and refinement of community recovery in sparse network models using spectral clustering, one of the most popular methods in the literature and in practice. The second topic is network community detection in a statistical minimax framework, including information-theoretic lower bounds quantified by a comprehensive collection of model parameters, and optimal estimation procedures that achieve the lower bounds. The third topic is goodness-of-fit tests for general sparse principal components analysis models, where adaptive procedures will be developed using a detection boundary framework; and the high dimensionality challenge will be tackled by considering regular alternatives such as Sobolev ellipsoids.