CIF: SMALL: The Linear Information Coupling Problem

在本研究中，我们研究一个新的问题设置在网络上传输信息，我们称之为线性信息耦合问题。我们不是问通过给定的信道可以传送多少信息比特，而是问如何有效地发送一层薄薄的信息。除了它的操作影响，以及它所解决的新应用，我们指出，这个新的公式是一个通用的和根本的简化网络容量的问题，这一直是开放的几十年。我们观察到，许多网络信息论问题的主要困难本质上是相同的：概率分布上的高维优化问题没有足够的结构。线性耦合问题的关键步骤是Kullback-Leibler发散的局部二次近似，由此定义了概率分布空间的几何结构。这有助于我们将信息转换可视化为更简单的几何操作，例如关于正交基的投影和扩展，从而确定有效的信息传递方式。配备了这种新的分析工具，本项目研究了两类问题。首先，正如我们的初步结果所证明的那样，局部近似在解决一些开放网络通信问题方面特别强大。我们推广这些结果，构建一个新的网络模型，其中每个连接是线性化的，其特征在于相应的奇异值分解（SVD）结构。利用该模型，通常可以显式地求解最优网络操作和相应的性能。其次，我们扩展这种方法来研究更一般的信息交换，包括实时，动态和容易出错的系统。这使我们能够将信息论的原理扩展到比传统编码数字应用更广泛的背景。

In this research, we study a new problem setting in transmitting information over networks, which we call the linear information coupling problem. Instead of asking how many information bits can be conveyed through a given channel, we ask how can one efficiently send a thin layer of information. Aside from its operational implications, and the new applications it addresses, we point out that this new formulation is a generic and fundamental simplification to the network capacity problem, which has remained open for decades. We observe that the main difficulties in many network information theory problems are essentially the same: the high dimensional optimization problems over probability distributions do not have sufficient structure. The key step of the linear coupling problems is a local quadratic approximation of the Kullback-Leibler divergence, from where a geometric structure for the space of probability distributions is defined. This helps us to visualize the information transitions as simpler geometric operations such as projections and expansion with respect to orthonormal bases, and thus identify efficient ways to convey information.Equipped with this new analysis tool, this project studies two classes of problems. First, as demonstrated with our preliminary results, the local approximation is particularly powerful in solving some open network communication problems. We generalize these results to construct a new network model, where each connection is linearized and characterized by the corresponding singular value decomposition (SVD) structure. With this model, the optimal network operations and the corresponding performance can often be solved explicitly. Secondly, we extend this approach to study more general information exchanges including real-time, dynamic, and error-prone systems. This allows us extend the principle of information theory to a much broader context than conventional coded digital applications.