CIF: SMALL: The Linear Information Coupling Problem

在本研究中，我们研究了一种新的网络信息传输问题，我们称之为线性信息耦合问题。我们问的不是通过一个给定的信道可以传递多少信息位，而是如何有效地发送一层薄薄的信息。除了它的操作意义和它所解决的新应用之外，我们指出，这个新公式是对网络容量问题的一个通用的和基本的简化，这个问题几十年来一直没有解决。我们观察到，许多网络信息理论问题的主要困难本质上是相同的：概率分布上的高维优化问题没有足够的结构。线性耦合问题的关键步骤是Kullback-Leibler散度的局部二次逼近，并由此定义概率分布空间的几何结构。这有助于我们将信息转换可视化为更简单的几何运算，例如关于标准正交基的投影和展开，从而确定传达信息的有效方法。利用这个新的分析工具，本项目研究了两类问题。首先，正如我们的初步结果所证明的那样，局部近似在解决一些开放网络通信问题方面特别强大。我们将这些结果推广到一个新的网络模型，其中每个连接都被线性化并用相应的奇异值分解（SVD）结构来表征。利用该模型，通常可以明确地求解网络的最优操作和相应的性能。其次，我们将此方法扩展到更一般的信息交换，包括实时、动态和易出错的系统。这使我们能够将信息论的原理扩展到比传统编码数字应用更广泛的背景下。

In this research, we study a new problem setting in transmitting information over networks, which we call the linear information coupling problem. Instead of asking how many information bits can be conveyed through a given channel, we ask how can one efficiently send a thin layer of information. Aside from its operational implications, and the new applications it addresses, we point out that this new formulation is a generic and fundamental simplification to the network capacity problem, which has remained open for decades. We observe that the main difficulties in many network information theory problems are essentially the same: the high dimensional optimization problems over probability distributions do not have sufficient structure. The key step of the linear coupling problems is a local quadratic approximation of the Kullback-Leibler divergence, from where a geometric structure for the space of probability distributions is defined. This helps us to visualize the information transitions as simpler geometric operations such as projections and expansion with respect to orthonormal bases, and thus identify efficient ways to convey information.Equipped with this new analysis tool, this project studies two classes of problems. First, as demonstrated with our preliminary results, the local approximation is particularly powerful in solving some open network communication problems. We generalize these results to construct a new network model, where each connection is linearized and characterized by the corresponding singular value decomposition (SVD) structure. With this model, the optimal network operations and the corresponding performance can often be solved explicitly. Secondly, we extend this approach to study more general information exchanges including real-time, dynamic, and error-prone systems. This allows us extend the principle of information theory to a much broader context than conventional coded digital applications.