Learning-Oriented Data Compression with Applications

随着移动设备数量的不断增加，以及监控系统和传感器网络的广泛部署，正在产生和收集前所未有的大量数据。数据压缩技术在应对存储、传输、处理和分析此类数据的重大挑战方面发挥着越来越重要的作用。本研究旨在发展一种面向学习的数据压缩理论，研究相关的算法设计，并探索这种新的数据压缩范式的实际应用。它代表了开发一个成熟的压缩感知学习理论的长期目标的踏脚石。本文提出的研究包括集中压缩、分布式压缩和广义分布式压缩三个方向。在面向学习的集中式压缩系统中，将源编码成位串，解码器根据该位串生成满足规定约束的目标数据的相关特征的概率描述。基于一种新的优化技术，将开发一种统一的方法来估计基本压缩极限并生成最优特征表示。基于这种方法，将构建用于视频分析的紧凑描述符。这项工作也有可能为深度神经网络的设计和压缩提供启示。面向学习的分布式压缩通过对不相交的源组件进行单独编码来推广其集中式压缩。对于这一问题，基于信息几何的传统方法仅在低速率极限下有效。由于问题的非凸性，在高速率区域出现了显著的分析困难。将引入先进的分析技术来解决这些困难。在许多其他应用中，所得理论将用于指导云无线接入网络中信令和量化子系统的联合设计。面向学习的广义分布式数据压缩是分布式数据压缩的进一步扩展，它允许源组件和编码器之间的任意连接。面向学习的广义分布式数据压缩理论一旦完成，就可以用来设计一种增强的预测编码体系结构，该体系结构能够达到非因果有损压缩的性能极限。本文的研究将极大地丰富数据压缩理论，拓宽数据压缩的应用范围。参与本研究计划的学生将在理论建构、演算法设计及系统实作等方面，接受均衡且全面的训练。他们将获得的洞察力的广度将使他们成为这个新兴研究领域的领导者。

With the ever-growing number of mobile devices, as well as the widespread deployment of surveillance systems and sensor networks, an unprecedented amount of data is being generated and collected. Data compression techniques are playing an increasingly important role in meeting the significant challenges in storing, transmitting, processing, and analyzing such data. The proposed research aims to develop a theory of learning-oriented data compression, investigate the relevant algorithm design, and explore the practical applications of this new data compression paradigm. It represents a stepping stone in the long-term goal of developing a full-fledged compression-aware learning theory.The proposed research consists of three thrusts: centralized compression, distributed compression, and generalized distributed compression.In a learning-oriented centralized compression system, a source is encoded into a bit string, and based on that string, the decoder produces a probabilistic description of the relevant features of the target data satisfying the prescribed constraints. A unified approach will be developed based on a novel optimization technique for estimating the fundamental compression limits and generating the optimal feature representations. Compact descriptors for video analysis will be constructed based on this approach. This line of work also has the potential of shedding light on the design and compression of deep neural networks. Learning-oriented distributed compression generalizes its centralized counterpart by separately encoding disjoint source components. For this problem, traditional approaches based on information geometry are only effective in the low-rate limit. Significant analytical difficulties arise in the high-rate region due to the non-convex nature of the problem. Advanced analytical techniques will be introduced to tackle these difficulties. Among many other applications, the resulting theory will be used to guide the joint design of the signaling and quantizing subsystems in cloud radio access networks.Learning-oriented generalized distributed data compression is a further extension of distributed data compression that allows arbitrary connections between the source components and the encoders. The theory of learning-oriented generalized distributed data compression, once completed, can be leveraged to design an enhanced predictive coding architecture that is able to achieve the performance limit of noncausal lossy compression.The proposed research will greatly enrich the theory of data compression and broaden the scope of its applications. The students involved in this research program will receive a balanced and comprehensive training in theory building, algorithm design, and system implementation. The breadth of the insight that they will acquire will enable them to become leaders of this emerging research field.