Spectral Methods for Data Science: A Statistical Perspective
Spectral Methods for Data Science: A Statistical Perspective
复制标题
DOI：
10.1561/2200000079
复制
发表时间：
2021-01-01
影响因子：
32.8
通讯作者：
Ma, Cong
中科院分区：
文献类型：
--
作者：
Chen, Yuxin;Chi, Yuejie;Ma, Cong
光谱方法已经成为一种简单但令人惊讶的有效方法，用于从大量，嘈杂和不完整的数据中提取信息。简而言之，谱方法指的是建立在特征值（分别为）上的算法集合。奇异值）和特征向量（分别奇异向量）的一些适当设计的矩阵构造的数据。在机器学习、成像科学、金融和计量经济学建模以及信号处理中已经发现了各种各样的应用，包括推荐系统、社区检测、排名、结构化矩阵恢复、张量数据估计、联合形状匹配、盲解卷积、金融投资、风险管理、治疗评估、因果推断等。谱方法由于其简单有效，不仅被用作独立的估计器，而且经常被用来促进其他更复杂的算法以提高性能。虽然谱方法的研究可以追溯到经典的矩阵摄动理论和矩量法，但过去十年来，通过统计建模的透镜，借助于浓度不等式和非渐近随机矩阵理论。这本专著的目的是提出一个系统的，全面的，但从现代统计学的角度介绍光谱方法，突出其算法在不同的大规模应用的影响。特别是，我们的论述围绕着几个跨越各种应用的中心问题：如何表征光谱方法在达到目标统计精度水平时的样本效率，以及如何评估它们在面对随机噪声，缺失数据和对抗性腐败时的稳定性？除了传统的l（2）扰动分析，我们提出了一个系统的l（无穷大）和l（2，无穷大）扰动理论的特征空间和奇异子空间，这是最近才成为可用的，由于一个强大的“留一”的分析框架。
Spectral methods have emerged as a simple yet surprisingly effective approach for extracting information from massive, noisy and incomplete data. In a nutshell, spectral methods refer to a collection of algorithms built upon the eigenvalues (resp. singular values) and eigenvectors (resp. singular vectors) of some properly designed matrices constructed from data. A diverse array of applications have been found in machine learning, imaging science, financial and econometric modeling, and signal processing, including recommendation systems, community detection, ranking, structured matrix recovery, tensor data estimation, joint shape matching, blind deconvolution, financial investments, risk managements, treatment evaluations, causal inference, amongst others. Due to their simplicity and effectiveness, spectral methods are not only used as a stand-alone estimator, but also frequently employed to facilitate other more sophisticated algorithms to enhance performance.While the studies of spectral methods can be traced back to classical matrix perturbation theory and the method of moments, the past decade has witnessed tremendous theoretical advances in demystifying their efficacy through the lens of statistical modeling, with the aid of concentration inequalities and non-asymptotic random matrix theory. This monograph aims to present a systematic, comprehensive, yet accessible introduction to spectral methods from a modern statistical perspective, highlighting their algorithmic implications in diverse large-scale applications. In particular, our exposition gravitates around several central questions that span various applications: how to characterize the sample efficiency of spectral methods in reaching a target level of statistical accuracy, and how to assess their stability in the face of random noise, missing data, and adversarial corruptions? In addition to conventional l(2) perturbation analysis, we present a systematic l(infinity) and l(2,infinity) perturbation theory for eigenspace and singular subspaces, which has only recently become available owing to a powerful "leave-one-out" analysis framework.