RI: Small: Deep Variational Data Compression

没有数据压缩，互联网和世界上的IT系统就不可能存在。从大型商业数据库的高效存储到大型强子对撞机收集的海量数据集，再到视频流，压缩是一种至关重要的工具，它使我们的社会所依赖的许多系统得以实现。据估计，到2021年，仅压缩视频数据就将占互联网流量的80%以上（这是在COVID-19之前做出的估计）。视频编码效率的任何提高都将产生巨大的社会影响。在过去的几年里，很明显，神经网络可以显着改善经典的压缩方法，在如何权衡文件大小的数据质量损失方面。除此之外，神经压缩方法还有其他好处：它们可以微调到特定的数据模态（例如，医学图像）不显示常见的块编码视觉伪像，并且可以被“监督”以将更多注意力分配给感兴趣的特定特征。该奖项通过改进视频编码，实现机器学习系统之间更快的数据传输，以及提高神经编解码器设计的模块化程度，为更好的压缩算法做出贡献，潜在地影响广泛的应用。该项目借鉴了深度潜变量建模，推动了神经数据压缩的几个新想法：（i）分层生成视频编码;（ii）监督压缩，以及（iii）经训练的生成模型的即插即用压缩。部分（i）提出将联合收割机归一化流与顺序变分自编码器组合以预测具有更高置信度和更短预期码长度的未来帧。第（ii）部分利用深度神经网络的能力来训练多个任务，例如数据重建和分类。第（iii）部分描述了一种全新的方法，该方法将离散化与训练相结合，而是联合执行离散化和熵编码。该算法考虑了后验不确定性，将更多的比特分配给对重建给定输入数据点最重要的特征，同时将较少的比特分配给可以容忍量化误差的特征。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

The internet and the world’s IT systems could not exist without data compression. From the efficient storage of large business databases to massive datasets collected by the Large Hadron Collider to video streaming—compression is a tool of fundamental importance that enables many of the systems our societies have come to depend on. It is estimated that by 2021, compressed video data alone will account for over 80% of internet traffic (an estimate made before COVID-19). Any gains that can be made in video coding efficiency will have a dramatic societal impact. Over the past few years, it has become clear that neural networks can significantly improve classical compression methods in terms of how they trade off data quality losses for file size. Besides this, neural compression methods also have other benefits: they can be fine-tuned to specific data modalities (e.g., medical images), do not show the common block-coding visual artifacts, and can be ‘supervised’ to allocate more attention to specific features of interest. This award contributes to better compression algorithms by improving video coding, enabling faster data transfer between machine learning systems, and improving the modularity of neural codec design, potentially impacting a wide range of applications.This project draws on deep latent variable modeling and promotes several new ideas for neural data compression: (i) hierarchical generative video coding; (ii) supervised compression, and (iii) plug and play compression of trained generative models. Part (i) proposes to combine normalizing flows with sequential variational autoencoders to predict future frames with higher confidence and shorter expected code lengths. Part (ii) leverages the ability of deep neural networks to be trained towards multiple tasks, such as data reconstruction and classification. Part (iii) describes a fundamentally new approach that decouples discretization from training, and that instead performs discretization and entropy coding jointly. The algorithm takes posterior uncertainties into account to allocate more bits to the features that are most important to reconstruct a given input data point while assigning fewer bits to features where some quantization error can be tolerated.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.