Information Theory with Directions: Geometric Structure and Coordinates on the Space of Probability Distributions

在这项研究中，我们研究了一组几何方法，可以用来从根本上扩展我们的信息理论分析的方法。研究信息传输的传统方法是基于熵和互信息等关键量，它们可以被视为分布之间的距离。这种方法在研究静态点对点通信问题时特别有用，其中涉及的概率分布相对较少。面对理解大型动态无线网络的挑战，由于经常涉及许多高维分布，传统的方法在解决甚至描述问题时往往变得繁琐。在这项工作中研究的几何方法是在某种意义上？微积分？分布的空间。通过发展概念的内部产品，预测，并在空间的坐标系的分布，我们增加了一种感觉？方向？在信息论分析中。这种方法的新的见解往往导致更好地理解和扩展现有的网络信息理论的结果。几何方法主要用于两组问题。首先，通过可视化多个分布之间的关系，我们发展了发散转移的概念来描述通过一般统计耦合的信息交换。这种方法在研究多终端通信问题，描述和控制不同终端的信息内容方面特别有用。其次，我们使用这种方法来研究在动态通信信道中的错误保护。传统观点认为，所有信息都可以转换为比特，并在传输中得到统一保护，这显然不适合动态网络应用。在异构信息处理和不等错误保护的新概念的研究中，几何方法起着至关重要的作用。

In this research, we study a set of geometric approaches that can be used to fundamentally extend our methodology of information theoretical analysis. The conventional approaches to study information transmissions are based on the key quantities like entropy and mutual information, which can be viewed as a distance between distributions. Such approaches are particularly useful in studying static point-to-point communication problems, where relatively few probability distributions are involved. Facing the challenge of understanding large dynamic wireless networks, as many distributions with high dimensionalities are often involved, the conventional approaches often become cumbersome in solving or even describing the problems. The geometric approach studied in this work is in a sense a ?calculus? on the space of distributions. By developing notions of inner products, projections, and coordinate systems in the space of distributions, we add a sense of ?direction? in information theoretic analysis. The new insights from this approach often lead to better understanding and extensions to the existing network information theory results.The geometric approach is mainly used in two sets of problems. First, by visualizing the relation between multiple distributions, we develop the notion of divergence transition to describe the information exchange through general statistical coupling. This approach is particularly useful in studying mutli-terminal communication problems, in describing and controlling information contents at different terminals. Secondly, we used this approach to study error protections in dynamic communication channels. The conventional wisdom suggests that all information can be converted into bits and uniformly protected in transmissions, which is clearly inadequate for dynamic networked applications. In the study of new notions of heterogeneous information processing and unequal error protections, the geometric approach plays a crucial role.