Learning-Oriented Data Compression with Applications

随着移动设备数量的不断增加，以及监控系统和传感器网络的广泛部署，正在产生和收集前所未有的数据量。数据压缩技术在应对存储、传输、处理和分析此类数据方面的重大挑战方面发挥着越来越重要的作用。本研究旨在发展一种面向学习的数据压缩理论，研究相关的算法设计，并探索这种新的数据压缩范式的实际应用。它代表了发展成熟的压缩感知学习理论的长期目标的垫脚石。*所提出的研究包括三个方面：集中式压缩、分布式压缩和广义分布式压缩。*在面向学习的集中式压缩系统中，信源被编码成比特串，解码器基于该比特串产生满足规定约束的目标数据的相关特征的概率描述。将基于一种新的优化技术开发一种统一的方法，用于估计基本压缩极限和生成最优特征表示。基于这种方法，将构建用于视频分析的紧凑描述符。这项工作还有可能为深度神经网络的设计和压缩带来启发。*面向学习的分布式压缩通过单独编码不相交的源组件来概括其集中式对应的压缩。对于这个问题，传统的基于信息几何的方法只在低码率极限下有效。由于问题的非凸性，在高速率区域出现了很大的分析困难。将采用先进的分析技术来解决这些困难。在许多其他应用中，所得到的理论将用于指导云无线电接入网络中信令和量化子系统的联合设计。*面向学习的广义分布式数据压缩是分布式数据压缩的进一步扩展，它允许源组件和编码器之间的任意连接。面向学习的广义分布式数据压缩理论一旦完成，就可以用来设计一种能够达到非因果有损压缩性能极限的增强型预测编码结构。参与这项研究计划的学生将在理论构建、算法设计和系统实现方面接受平衡和全面的培训。他们将获得的洞察力的广度将使他们成为这一新兴研究领域的领导者。

With the ever-growing number of mobile devices, as well as the widespread deployment of surveillance systems and sensor networks, an unprecedented amount of data is being generated and collected. Data compression techniques are playing an increasingly important role in meeting the significant challenges in storing, transmitting, processing, and analyzing such data. The proposed research aims to develop a theory of learning-oriented data compression, investigate the relevant algorithm design, and explore the practical applications of this new data compression paradigm. It represents a stepping stone in the long-term goal of developing a full-fledged compression-aware learning theory.******The proposed research consists of three thrusts: centralized compression, distributed compression, and generalized distributed compression.******In a learning-oriented centralized compression system, a source is encoded into a bit string, and based on that string, the decoder produces a probabilistic description of the relevant features of the target data satisfying the prescribed constraints. A unified approach will be developed based on a novel optimization technique for estimating the fundamental compression limits and generating the optimal feature representations. Compact descriptors for video analysis will be constructed based on this approach. This line of work also has the potential of shedding light on the design and compression of deep neural networks. ******Learning-oriented distributed compression generalizes its centralized counterpart by separately encoding disjoint source components. For this problem, traditional approaches based on information geometry are only effective in the low-rate limit. Significant analytical difficulties arise in the high-rate region due to the non-convex nature of the problem. Advanced analytical techniques will be introduced to tackle these difficulties. Among many other applications, the resulting theory will be used to guide the joint design of the signaling and quantizing subsystems in cloud radio access networks.******Learning-oriented generalized distributed data compression is a further extension of distributed data compression that allows arbitrary connections between the source components and the encoders. The theory of learning-oriented generalized distributed data compression, once completed, can be leveraged to design an enhanced predictive coding architecture that is able to achieve the performance limit of noncausal lossy compression.******The proposed research will greatly enrich the theory of data compression and broaden the scope of its applications. The students involved in this research program will receive a balanced and comprehensive training in theory building, algorithm design, and system implementation. The breadth of the insight that they will acquire will enable them to become leaders of this emerging research field.