A linearly convergent algorithm for distributed principal component analysis
A linearly convergent algorithm for distributed principal component analysis
复制标题
DOI：
10.1016/j.sigpro.2021.108408
复制
发表时间：
2022-04-01
影响因子：
4.4
通讯作者：
Bajwa, Waheed U.
中科院分区：
文献类型：
--
作者：
Gang, Arpita;Bajwa, Waheed U.
主成分分析（PCA）是大数据时代降维的主力工具。虽然经常被忽视，但PCA的目的不仅是降低数据维度，而且还产生不相关的特征。此外，现代世界中不断增加的数据量通常需要在多台机器上存储数据样本，这排除了使用集中式PCA算法。本文重点研究PCA的双重目标，即降维和去相关的功能，但在分布式设置。这需要估计数据协方差矩阵的特征向量，而不是当数据分布在机器网络中时仅估计特征向量所跨越的子空间。虽然最近已经提出了一些分布式解决方案的PCA问题，收敛保证和/或这些解决方案的通信开销仍然是一个问题。着眼于通信效率，本文介绍了一种前馈神经网络为基础的单时标分布式PCA算法称为分布式桑格算法（DSA），估计的特征向量的数据协方差矩阵时，数据分布在一个无向和任意连接的网络的机器。此外，所提出的算法线性收敛到真解的邻域。数值结果也证明了所提出的解决方案的有效性。(c)2021爱思唯尔有限公司版权所有。
Principal Component Analysis (PCA) is the workhorse tool for dimensionality reduction in this era of big data. While often overlooked, the purpose of PCA is not only to reduce data dimensionality, but also to yield features that are uncorrelated. Furthermore, the ever-increasing volume of data in the modern world often requires storage of data samples across multiple machines, which precludes the use of centralized PCA algorithms. This paper focuses on the dual objective of PCA, namely, dimensionality reduction and decorrelation of features, but in a distributed setting. This requires estimating the eigenvectors of the data covariance matrix, as opposed to only estimating the subspace spanned by the eigenvectors, when data is distributed across a network of machines. Although a few distributed solutions to the PCA problem have been proposed recently, convergence guarantees and/or communications overhead of these solutions remain a concern. With an eye towards communications efficiency, this paper introduces a feedforward neural network-based one time-scale distributed PCA algorithm termed Distributed Sanger's Algorithm (DSA) that estimates the eigenvectors of the data covariance matrix when data is distributed across an undirected and arbitrarily connected network of machines. Furthermore, the proposed algorithm is shown to converge linearly to a neighborhood of the true solution. Numerical results are also provided to demonstrate the efficacy of the proposed solution.(c) 2021 Elsevier B.V. All rights reserved.