FINITE SAMPLE APPROXIMATION RESULTS FOR PRINCIPAL COMPONENT ANALYSIS: A MATRIX PERTURBATION APPROACH
FINITE SAMPLE APPROXIMATION RESULTS FOR PRINCIPAL COMPONENT ANALYSIS: A MATRIX PERTURBATION APPROACH
复制标题
DOI：
10.1214/08-aos618
复制
发表时间：
2008-12-01
影响因子：
4.5
通讯作者：
Nadler, Boaz
中科院分区：
文献类型：
--
作者：
Nadler, Boaz
主成分分析（PCA）是一种标准工具，用于对n组观测值（样本）进行降维，每个样本都有p个变量。本文利用矩阵摄动方法，研究了有限样本n上主成分分析的特征值与特征向量的非渐近关系，以及n ->∞下的极限种群主成分分析的特征值与特征向量的非渐近关系。与机器学习一样，我们提出了一个有限样本定理，该定理在尖峰协方差模型下，样本主成分分析和总体主成分分析的首特征值和特征向量之间的密切性具有很高的概率。此外，我们还考虑了有限样本PCA与联合极限p， n ->∞,p/n = c的渐近结果之间的关系。我们给出了“相变现象”的矩阵摄动观点，并基于简单的线性代数推导了该渐近极限的特征值和特征向量重叠。此外，我们的分析也适用于有限p， n，其中我们表明，尽管在无限情况下没有明显的相变，无论是作为噪声水平的函数还是作为样本量n的函数，样本PICA的特征向量可能会表现出明显的“跟踪损失”，突然失去其与总体PCA矩阵的（真实）特征向量的关系。这是由于信号的特征值与噪声的最大特征值之间的交叉，噪声的特征向量指向随机方向。
Principal component analysis (PCA) is a standard tool for dimensional reduction of a set of n observations (samples), each with p variables. In this paper, using a matrix perturbation approach, we study the nonasymptotic relation between the eigenvalues and eigenvectors of PCA computed on a finite sample of size n, and those of the limiting population PCA as n -> infinity. As in machine learning, we present a finite sample theorem which holds with high probability for the closeness between the leading eigenvalue and eigenvector of sample PCA and population PCA under a spiked covariance model. In addition, we also consider the relation between finite sample PCA and the asymptotic results in the joint limit p, n -> infinity, with p/n = c. We present a matrix perturbation view of the "phase transition phenomenon," and a simple linear-algebra based derivation of the eigenvalue and eigenvector overlap in this asymptotic limit. Moreover, our analysis also applies for finite p, n where we show that although there is no sharp phase transition as in the infinite case, either as a function of noise level or as a function of sample size n, the eigenvector of sample PICA may exhibit a sharp "loss of tracking," suddenly losing its relation to the (true) eigenvector of the population PCA matrix. This occurs due to a crossover between the eigenvalue due to the signal and the largest eigenvalue due to noise, whose eigenvector points in a random direction.