Neural Networks Optimally Compress the Sawbridge
Neural Networks Optimally Compress the Sawbridge
复制标题
DOI：
10.1109/dcc50243.2021.00022
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Aaron B. Wagner;Johannes Ball'e
中科院分区：
文献类型：
--
作者：
Aaron B. Wagner;Johannes Ball'e
基于神经网络的压缩器已被证明在压缩源（如图像）方面非常有效，这些源名义上是高维的，但假定集中在低维流形上。我们考虑一个连续时间的随机过程，模型的极端版本，这样的源，其中的实现下降沿着一维“曲线”的函数空间，具有无限维的线性跨度。我们精确地描述了这个源的最佳熵失真权衡，并在数值上表明，它是通过随机梯度下降训练的基于神经网络的压缩器实现的。相比之下，我们的分析和实验表明，压缩机的基础上经典的Karhunen-Loeve变换是非常次优的高速率。
Neural-network-based compressors have proven to be remarkably effective at compressing sources, such as images, that are nominally high-dimensional but presumed to be concentrated on a low-dimensional manifold. We consider a continuous-time random process that models an extreme version of such a source, wherein the realizations fall along a one-dimensional “curve” in function space that has infinite-dimensional linear span. We precisely characterize the optimal entropy-distortion tradeoff for this source and show numerically that it is achieved by neural-network-based compressors trained via stochastic gradient descent. In contrast, we show both analytically and experimentally that compressors based on the classical Karhunen-Loeve transform are highly suboptimal at high rates.