How to reduce dimension with PCA and random projections?
How to reduce dimension with PCA and random projections?
复制标题
DOI：
10.1109/tit.2021.3112821
复制
发表时间：
2021-12
影响因子：
2.5
通讯作者：
Woodruff, David P.
中科院分区：
文献类型：
--
作者：
Yang, Fan;Liu, Sifan;Dobriban, Edgar;Woodruff, David P.
关键词：
在我们的“大数据”时代，数据的规模和复杂性正在稳步增加。降维的方法越来越流行和有用。两种不同类型的降维方法是“数据无关”方法（如随机投影和草图）和“数据感知”方法（如主成分分析（PCA））。两者都有各自的优势，例如随机投影的速度和PCA的数据自适应性。在这项工作中，我们研究如何将它们联合收割机结合起来，以达到两者的最佳效果。我们研究“草图和解决”的方法，采取随机投影（或草图）第一，并计算PCA后。我们计算几种流行的素描方法（随机iid投影，随机采样，二次采样Hadamard变换，CountSketch等）在一般的“信号加噪声”（或尖峰）数据模型的性能。与已知的工作相比，我们的结果（1）给出了渐近精确的结果，（2）适用于信号分量仅略高于噪声，但投影维数不可忽略。我们还研究了更强的信号，允许更一般的协方差结构。我们发现，（a）根据数据的结构和绘制方法，投影下的信号强度以一种微妙的方式降低，（B）正交投影稍微更准确，（c）由于测量的集中，随机化不会造成太大的伤害，（d）CountSketch可以通过归一化方法得到一定程度的改善。我们的研究结果对统计学习和数据分析具有重要意义。我们还说明，结果是高度准确的模拟和分析经验数据。
In our “big data” age, the size and complexity of data is steadily increasing. Methods for dimension reduction are ever more popular and useful. Two distinct types of dimension reduction are “data-oblivious” methods such as random projections and sketching, and “data-aware” methods such as principal component analysis (PCA). Both have their strengths, such as speed for random projections, and data-adaptivity for PCA. In this work, we study how to combine them to get the best of both. We study “sketch and solve” methods that take a random projection (or sketch) first, and compute PCA after. We compute the performance of several popular sketching methods (random iid projections, random sampling, subsampled Hadamard transform, CountSketch, etc) in a general “signal-plus-noise” (or spiked) data model. Compared to well-known works, our results (1) give asymptotically exact results, and (2) apply when the signal components are only slightly above the noise, but the projection dimension is non-negligible. We also study stronger signals allowing more general covariance structures. We find that (a) signal strength decreases under projection in a delicate way depending on the structure of the data and the sketching method, (b) orthogonal projections are slightly more accurate, (c) randomization does not hurt too much, due to concentration of measure, (d) CountSketch can be somewhat improved by a normalization method. Our results have implications for statistical learning and data analysis. We also illustrate that the results are highly accurate in simulations and in analyzing empirical data.
登录
查看更多内容
影响因子：
1.4
作者：
Bloemendal, Alex;Erdos, Laszlo;Yin, Jun
通讯作者：
Yin, Jun
影响因子：
1.8
作者：
Cordero-Grande, Lucilio
通讯作者：
Cordero-Grande, Lucilio
影响因子：
32.8
作者：
Burges, Christopher J. C.
通讯作者：
Burges, Christopher J. C.
影响因子：
1.6
作者：
Baik, Jinho;Silverstein, Jack W.
通讯作者：
Silverstein, Jack W.
影响因子：
1.7
作者：
Anderson, Greg W.;Farrell, Brendan
通讯作者：
Farrell, Brendan