A Characterization of Entropy in Terms of Information Loss
A Characterization of Entropy in Terms of Information Loss
复制标题
DOI：
10.3390/e13111945
复制
发表时间：
2011-11-01
期刊：
影响因子：
2.7
通讯作者：
Leinster, Tom
中科院分区：
文献类型：
--
作者：
Baez, John C.;Fritz, Tobias;Leinster, Tom
香农熵和Tsallis熵有许多表征，作为服从某些性质的信息的度量。使用Faddeev和Furuichi的工作，我们得到一个非常简单的表征。这个特征不是集中在有限集上概率测度的熵上，而是集中在与测度保持函数相关的“信息损失”或熵的变化上。信息损失是条件熵的一种特殊情况：也就是说，它是一个随机变量的熵，条件是该变量的某个函数。我们表明，香农熵给出了唯一的概念，是函的，凸线性和连续的信息损失。这个特征自然也推广到Tsallis熵。
There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the entropy of a probability measure on a finite set, this characterization focuses on the "information loss", or change in entropy, associated with a measure-preserving function. Information loss is a special case of conditional entropy: namely, it is the entropy of a random variable conditioned on some function of that variable. We show that Shannon entropy gives the only concept of information loss that is functorial, convex-linear and continuous. This characterization naturally generalizes to Tsallis entropy as well.