CIF: SMALL: The Linear Information Coupling Problem

CIF：小：线性信息耦合问题

• 批准号: 1216476
• 负责人: Lizhong Zheng
• 金额: $ 35万
• 依托单位: Massachusetts Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-07-01 至 2015-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1216476&HistoricalAwards=false
• 关键词: CIF; SMALL; Linear; Information; Coupling

In this research, we study a new problem setting in transmitting information over networks, which we call the linear information coupling problem. Instead of asking how many information bits can be conveyed through a given channel, we ask how can one efficiently send a thin layer of information. Aside from its operational implications, and the new applications it addresses, we point out that this new formulation is a generic and fundamental simplification to the network capacity problem, which has remained open for decades. We observe that the main difficulties in many network information theory problems are essentially the same: the high dimensional optimization problems over probability distributions do not have sufficient structure. The key step of the linear coupling problems is a local quadratic approximation of the Kullback-Leibler divergence, from where a geometric structure for the space of probability distributions is defined. This helps us to visualize the information transitions as simpler geometric operations such as projections and expansion with respect to orthonormal bases, and thus identify efficient ways to convey information.Equipped with this new analysis tool, this project studies two classes of problems. First, as demonstrated with our preliminary results, the local approximation is particularly powerful in solving some open network communication problems. We generalize these results to construct a new network model, where each connection is linearized and characterized by the corresponding singular value decomposition (SVD) structure. With this model, the optimal network operations and the corresponding performance can often be solved explicitly. Secondly, we extend this approach to study more general information exchanges including real-time, dynamic, and error-prone systems. This allows us extend the principle of information theory to a much broader context than conventional coded digital applications.

在这项研究中，我们研究了通过网络传输信息的新问题设置，我们称之为线性信息耦合问题。我们不问通过给定通道可以传送多少信息位，而是问如何有效地发送薄层信息。除了其操作影响及其解决的新应用之外，我们指出，这种新的表述是对网络容量问题的通用和根本简化，该问题几十年来一直存在。我们观察到，许多网络信息论问题的主要困难本质上是相同的：概率分布上的高维优化问题没有足够的结构。线性耦合问题的关键步骤是 Kullback-Leibler 散度的局部二次近似，从中定义概率分布空间的几何结构。这有助于我们将信息转换可视化为更简单的几何运算，例如相对于正交基的投影和扩展，从而确定传递信息的有效方法。借助这种新的分析工具，该项目研究两类问题。首先，正如我们的初步结果所证明的那样，局部近似在解决一些开放网络通信问题方面特别强大。我们概括这些结果来构建一个新的网络模型，其中每个连接都是线性化的，并通过相应的奇异值分解（SVD）结构来表征。通过该模型，通常可以明确地求解最佳网络操作和相应的性能。其次，我们将这种方法扩展到研究更一般的信息交换，包括实时、动态和容易出错的系统。这使我们能够将信息论原理扩展到比传统编码数字应用更广泛的背景。