CIF: Small: Compression for Learning over networks

CIF：小型：网络学习压缩

• 批准号: 2007714
• 负责人: Christina Fragouli
• 金额: $ 52.39万
• 依托单位: University of California-Los Angeles
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-08-01 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2007714&HistoricalAwards=false
• 关键词: CIF; Small; Compression; Learning; over

Data compression is a core component of all communication protocols, as it can translate to bandwidth savings, energy efficiency and low delay operations. In the traditional setup, an information source compresses its messages so that they can be communicated efficiently with the goal of ensuring accurate reconstruction at the destination. This project seeks to design compression schemes that are specifically tailored to Machine Learning applications: If the transmitted messages support a given learning task (e.g., classification or learning), the desired compression schemes should provide better support for the learning task instead of focusing on reconstruction accuracy. This approach to compression could potentially yield significant benefits in terms of communication efficiency, while simultaneously promoting the successful implementation of Machine Learning algorithms. By improving communication efficiency, such schemes are expected to contribute to the successful implementation of distributed machine learning algorithms over networks.Traditionally, compression schemes are evaluated using rate-distortion trade-offs; this project is interested in rate-accuracy trade-offs, where accuracy captures the effect that quantization may have on a specific machine learning task. There is particular interest in information-theoretic lower bounds and trade-offs, and in explicit compression for the following two questions: (1) How to compress for model training, when we need to use distributed communication constrained nodes to learn a model, fast and efficiently; and (2) How to compress for communication during inference. The project will derive bounds and algorithms for distributed compression of features coming from composite distributions that will be used for a machine learning task, such as classification. This work will advance the state of the art, and build new connections between the areas of data compression and distributed machine learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

数据压缩是所有通信协议的核心组成部分，因为它可以转化为带宽节省，能源效率和低延迟操作。在传统设置中，信息源压缩其消息，以便它们能够有效地通信，以确保在目的地精确重建。该项目旨在设计专门为机器学习应用量身定制的压缩方案：如果传输的消息支持给定的学习任务（例如，分类或学习），则所需的压缩方案应该为学习任务提供更好的支持，而不是专注于重建准确性。这种压缩方法在通信效率方面可能会产生显著的好处，同时促进机器学习算法的成功实现。通过提高通信效率，这些方案有望有助于在网络上成功实现分布式机器学习算法。传统上，压缩方案是使用率失真权衡来评估的；该项目对率-精度权衡感兴趣，其中精度捕获量化可能对特定机器学习任务产生的影响。人们对信息论的下界和权衡特别感兴趣，并对以下两个问题进行显式压缩：(1)当我们需要使用分布式通信约束节点来快速有效地学习模型时，如何压缩模型训练；(2)在推理过程中如何压缩以进行通信。该项目将为来自组合分布的特征的分布式压缩导出边界和算法，这些特征将用于机器学习任务，例如分类。这项工作将推动技术的发展，并在数据压缩和分布式机器学习领域之间建立新的联系。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Distributed User-Level Private Mean Estimation

• DOI: 10.1109/isit50566.2022.9834713
• 发表时间: 2022-06
• 期刊: 2022 IEEE International Symposium on Information Theory (ISIT)
• 作者: Antonious M. Girgis;Deepesh Data;S. Diggavi

Decentralized Learning Robust to Data Poisoning Attacks

去中心化学习对数据中毒攻击具有鲁棒性

• DOI: 10.1109/cdc51059.2022.9992702
• 发表时间: 2022
• 期刊: IEEE Control and Decision Conference (CDC
• 作者: Mao, Yanwen;Data, Deepesh;Diggavi, Suhas;Tabuada, Paulo
• 通讯作者: Tabuada, Paulo

On Coded Broadcasting for Wireless Recommendation Systems

无线推荐系统的编码广播

• DOI: 10.1109/icc42927.2021.9500829
• 发表时间: 2021
• 期刊: IEEE International Conference on Communications
• 作者: Vestergaard, Rasmus;Hanna, Osama;Song, Linqi;Lucani, Daniel E.;Fragouli, Christina
• 通讯作者: Fragouli, Christina

On the Generalized Degrees of Freedom of the Noncoherent Interference Channel

非相干干扰通道的广义自由度

• DOI: 10.1109/twc.2022.3153893
• 发表时间: 2022
• 期刊: IEEE Transactions on Wireless Communications
• 影响因子: 10.4
• 作者: Sebastian, Joyson;Diggavi, Suhas
• 通讯作者: Diggavi, Suhas

An entropy reduction approach to continual testing

持续测试的熵减少方法

• DOI: 10.1109/isit45174.2021.9518188
• 发表时间: 2021
• 期刊: Proceedings of the International Symposium on Information Theory (ISIT
• 作者: Srinivasavaradhan, Sundara Rajan;Nikolopoulos, Pavlos;Fragouli, Christina;Diggavi, Suhas
• 通讯作者: Diggavi, Suhas