Information Theory with Directions: Geometric Structure and Coordinates on the Space of Probability Distributions

在这项研究中，我们研究了一套几何方法，可以用来从根本上扩展我们的信息理论分析方法。传统的研究信息传递的方法是基于像信息熵和互信息这样的关键量，它可以被看作是分布之间的距离。这种方法在研究静态点对点通信问题时特别有用，在静态点对点通信问题中，涉及的概率分布相对较少。面对理解大规模动态无线网络的挑战，由于往往涉及到许多高维分布，传统的方法在求解甚至描述问题时往往变得繁琐。本文所研究的几何方法在某种意义上是微积分。在分布空间上。通过发展分布空间中的内积、投影和坐标系的概念，我们增加了方向感。在信息论分析中。这种方法的新见解往往会导致对现有网络信息理论结果的更好理解和扩展。几何方法主要用于两组问题。首先，通过可视化多个分布之间的关系，我们提出了散度跃迁的概念来描述通过一般统计耦合进行的信息交换。这种方法在研究多终端通信问题、描述和控制不同终端上的信息内容时特别有用。其次，我们利用这种方法研究了动态通信信道中的差错保护。传统观点认为，所有信息都可以转换为比特，并在传输中得到统一保护，这显然不适合动态联网应用。在研究异质信息处理和不等差错保护的新概念时，几何方法起着至关重要的作用。

In this research, we study a set of geometric approaches that can be used to fundamentally extend our methodology of information theoretical analysis. The conventional approaches to study information transmissions are based on the key quantities like entropy and mutual information, which can be viewed as a distance between distributions. Such approaches are particularly useful in studying static point-to-point communication problems, where relatively few probability distributions are involved. Facing the challenge of understanding large dynamic wireless networks, as many distributions with high dimensionalities are often involved, the conventional approaches often become cumbersome in solving or even describing the problems. The geometric approach studied in this work is in a sense a ?calculus? on the space of distributions. By developing notions of inner products, projections, and coordinate systems in the space of distributions, we add a sense of ?direction? in information theoretic analysis. The new insights from this approach often lead to better understanding and extensions to the existing network information theory results.The geometric approach is mainly used in two sets of problems. First, by visualizing the relation between multiple distributions, we develop the notion of divergence transition to describe the information exchange through general statistical coupling. This approach is particularly useful in studying mutli-terminal communication problems, in describing and controlling information contents at different terminals. Secondly, we used this approach to study error protections in dynamic communication channels. The conventional wisdom suggests that all information can be converted into bits and uniformly protected in transmissions, which is clearly inadequate for dynamic networked applications. In the study of new notions of heterogeneous information processing and unequal error protections, the geometric approach plays a crucial role.