Federated Neural Compression Under Heterogeneous Data
Federated Neural Compression Under Heterogeneous Data
复制标题
DOI：
10.1109/isit54713.2023.10206457
复制
发表时间：
2023-05
期刊：
影响因子：
--
通讯作者：
E. Lei;Hamed Hassani;S. S. Bidokhti-S.
中科院分区：
文献类型：
--
作者：
E. Lei;Hamed Hassani;S. S. Bidokhti-S.
我们讨论了一个联邦学习的压缩问题，其目标是从现实世界中学习一个散布在客户端的现实数据，并且可能是统计上异质的，但我们共享一个共同的基础表示。自然而然地提出了一种使用分析和综合转换客户启发的压缩机体系结构。方法，我们雇用了一个对每个客户的熵模型，这使得在客户端学习一个全球潜在空间，以及适应客户潜在分布的个性化熵模型。 ，这表明学习的压缩也从统计上异质的联合设置中的共享全球表示中受益。
We discuss a federated learned compression problem, where the goal is to learn a compressor from real-world data which is scattered across clients and may be statistically heterogeneous, yet share a common underlying representation. We propose a distributed source model that encompasses both characteristics, and naturally suggests a compressor architecture that uses analysis and synthesis transforms shared by clients. Inspired by personalized federated learning methods, we employ an entropy model that is personalized to each client. This allows for a global latent space to be learned across clients, and personalized entropy models that adapt to the clients’ latent distributions. We show empirically that this strategy outperforms solely local methods, which indicates that learned compression also benefits from a shared global representation in statistically heterogeneous federated settings.