INFLUENTIAL FEATURES PCA FOR HIGH DIMENSIONAL CLUSTERING
INFLUENTIAL FEATURES PCA FOR HIGH DIMENSIONAL CLUSTERING
复制标题
DOI：
10.1214/15-aos1423
复制
发表时间：
2016-12-01
影响因子：
4.5
通讯作者：
Wang, Wanjie
中科院分区：
文献类型：
--
作者：
Jin, Jiashun;Wang, Wanjie
我们考虑一个聚类问题，其中我们观察到特征向量X-i是R-P的元素，i = 1，2，.，n，从K个可能的类。类标签是未知的，主要的兴趣是估计它们。本文主要研究了p >> n的现代情况下经典聚类方法面临的挑战，提出了影响特征PCA（IF-PCA）作为一种新的聚类方法。在IF-PCA中，我们选择一小部分具有最大Kolmogorov Smirnov（KS）分数的特征，获得选择后归一化数据矩阵的第一个（K-1）左奇异向量，然后通过对这些奇异向量应用经典的k-means过程来估计标签。在该过程中，唯一的调整参数是特征选择步骤中的阈值。我们以数据驱动的方式设定了门槛，采用了最近的“更高批评”概念。因此，IF-PCA是一种无需调整的聚类方法。该方法在聚类方面具有较好的性能。特别是，在三个数据集，IF-PCA的错误率只有29%或更少的错误率的其他方法。我们还重新发现了Efron [J. Amer. Statistist. 99（2004）96-104]。通过精细的分析，特别是选择后特征分析，我们推导出Kolmogorov Smirnov统计量的严格概率界限，并表明IF-PCA在广泛的背景下产生聚类一致性。聚类问题与稀疏PCA和低秩矩阵恢复问题有关，但在重要方面有所不同。我们揭示了一个有趣的相变现象与这些问题，并确定每个感兴趣的范围。
We consider a clustering problem where we observe feature vectors X-i is an element of R-P, i = 1, 2,..., n, from K possible classes. The class labels are unknown and the main interest is to estimate them. We are primarily interested in the modern regime of p >> n, where classical clustering methods face challenges.We propose Influential Features PCA (IF-PCA) as a new clustering procedure. In IF-PCA, we select a small fraction of features with the largest Kolmogorov Smirnov (KS) scores, obtain the first (K-1) left singular vectors of the post-selection normalized data matrix, and then estimate the labels by applying the classical k-means procedure to these singular vectors. In this procedure, the only tuning parameter is the threshold in the feature selection step. We set the threshold in a data-driven fashion by adapting the recent notion of Higher Criticism. As a result, IF-PCA is a tuning-free clustering method.We apply IF-PCA to 10 gene microarray data sets. The method has competitive performance in clustering. Especially, in three of the data sets, the error rates of IF-PCA are only 29% or less of the error rates by other methods. We have also rediscovered a phenomenon on empirical null by Efron [J. Amer. Statist. Assoc. 99 (2004) 96-104] on microarray data.With delicate analysis, especially post-selection eigen-analysis, we derive tight probability bounds on the Kolmogorov Smirnov statistics and show that IF-PCA yields clustering consistency in a broad context. The clustering problem is connected to the problems of sparse PCA and low-rank matrix recovery, but it is different in important ways. We reveal an interesting phase transition phenomenon associated with these problems and identify the range of interest for each.