RI: Small: Deep Variational Data Compression

没有数据压缩，互联网和全球IT系统就不可能存在。从大型商业数据库的高效存储到大型强子对撞机收集的大量数据集，再到视频流，压缩是一种至关重要的工具，它使我们社会所依赖的许多系统得以实现。据估计，到2021年，仅压缩视频数据就将占到互联网流量的80%以上（这是新冠肺炎疫情前的估计）。视频编码效率的任何提高都将产生巨大的社会影响。在过去的几年里，神经网络在如何权衡数据质量损失和文件大小方面可以显著改进经典压缩方法已经变得很清楚。除此之外，神经压缩方法还有其他好处：它们可以针对特定的数据模式（例如，医学图像）进行微调，不会显示常见的块编码视觉伪像，并且可以“监督”以将更多注意力分配到感兴趣的特定特征上。该奖项通过改进视频编码，实现机器学习系统之间更快的数据传输，以及改进神经编解码器设计的模块化，从而有助于更好的压缩算法，潜在地影响广泛的应用。该项目借鉴了深度潜在变量建模，并提出了几个神经数据压缩的新思路：(i)分层生成视频编码；（ii）监督压缩，（iii）训练生成模型的即插即用压缩。第(i)部分提出将规范化流与顺序变分自编码器相结合，以更高的置信度和更短的期望码长预测未来的帧。第（ii）部分利用深度神经网络的能力进行多任务训练，如数据重建和分类。第（iii）部分描述了一种基本的新方法，该方法将离散化与训练解耦，而是联合执行离散化和熵编码。该算法考虑了后验不确定性，将更多的比特分配给对重构给定输入数据点最重要的特征，而将更少的比特分配给量化误差可以容忍的特征。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The internet and the world’s IT systems could not exist without data compression. From the efficient storage of large business databases to massive datasets collected by the Large Hadron Collider to video streaming—compression is a tool of fundamental importance that enables many of the systems our societies have come to depend on. It is estimated that by 2021, compressed video data alone will account for over 80% of internet traffic (an estimate made before COVID-19). Any gains that can be made in video coding efficiency will have a dramatic societal impact. Over the past few years, it has become clear that neural networks can significantly improve classical compression methods in terms of how they trade off data quality losses for file size. Besides this, neural compression methods also have other benefits: they can be fine-tuned to specific data modalities (e.g., medical images), do not show the common block-coding visual artifacts, and can be ‘supervised’ to allocate more attention to specific features of interest. This award contributes to better compression algorithms by improving video coding, enabling faster data transfer between machine learning systems, and improving the modularity of neural codec design, potentially impacting a wide range of applications.This project draws on deep latent variable modeling and promotes several new ideas for neural data compression: (i) hierarchical generative video coding; (ii) supervised compression, and (iii) plug and play compression of trained generative models. Part (i) proposes to combine normalizing flows with sequential variational autoencoders to predict future frames with higher confidence and shorter expected code lengths. Part (ii) leverages the ability of deep neural networks to be trained towards multiple tasks, such as data reconstruction and classification. Part (iii) describes a fundamentally new approach that decouples discretization from training, and that instead performs discretization and entropy coding jointly. The algorithm takes posterior uncertainties into account to allocate more bits to the features that are most important to reconstruct a given input data point while assigning fewer bits to features where some quantization error can be tolerated.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.