Communication-Efficient Distributed MAX-VAR Generalized CCA via Error Feedback-Assisted Quantization
Communication-Efficient Distributed MAX-VAR Generalized CCA via Error Feedback-Assisted Quantization
复制标题
DOI：
10.1109/icassp43922.2022.9746607
复制
发表时间：
2022-05
期刊：
影响因子：
--
通讯作者：
Sagar Shrestha;Xiao Fu
中科院分区：
文献类型：
--
作者：
Sagar Shrestha;Xiao Fu
广义典型相关分析（GCCA）旨在从数据的多个“视图”（例如，同一事件的音频和视频）。在大数据时代，GCCA计算遇到了许多新的挑战。特别是，GCCA的分布式优化-这是很好的动机在物联网和并行计算等应用程序-可能会产生过高的通信成本。为了解决这一问题，本文提出了一种通信效率高的分布式GCCA算法下流行的MAX-VAR GCCA范式。在该算法中，采用了一种量化策略，用于计算代理之间的信息交换。据观察，我们的设计，利用基于误差反馈的量化的想法，可以减少至少90%的通信成本，同时保持基本上相同的GCCA性能作为未量化的版本。此外，所提出的方法是guarante-anteed收敛到一个几何速率的最优解的邻域，即使在积极的量化。我们的方法的有效性证明使用合成和真实的数据实验。
Generalized canonical correlation analysis (GCCA) aims to learn common low-dimensional representations from multiple "views" of the data (e.g., audio and video of the same event). In the era of big data, GCCA computation encounters many new challenges. In particular, distributed optimization for GCCA—which is well-motivated in applications like internet of things and parallel computing—may incur prohibitively high communication costs. To address this challenge, this work proposes a communication-efficient distributed GCCA algorithm under the popular MAX-VAR GCCA paradigm. A quantization strategy for information exchange among the computing agents is employed in the proposed algorithm. It is observed that our design, leveraging the idea of error feedback-based quantization, can reduce communication cost by at least 90% while maintaining essentially the same GCCA performance as the unquantized version. Furthermore, the proposed method is guar-anteed to converge to a neighborhood of the optimal solution in a geometric rate—even under aggressive quantization. The effectiveness of our method is demonstrated using both synthetic and real data experiments.