Chained Kullback-Leibler Divergences.
Chained Kullback-Leibler Divergences.
复制标题
DOI：
10.1109/isit.2016.7541365
复制
发表时间：
2016-07
期刊：
影响因子：
--
通讯作者：
Weissman T
中科院分区：
文献类型：
--
作者：
Pavlichin DS;Weissman T
我们定义并表征了在所有中间分布 w 上最小化的“链式”Kullback-Leibler 散度 minw D(p‖w) + D(w‖q) 和在整个路径 (w1,…,wk−1) 上最小化的类似 k 倍链式 K-L 散度 min D(p‖wk−1) + … + D(w2‖w1) + D(w1‖q)。这个量出现在对一组类型的马尔可夫链的大偏差分析中——中性遗传漂变的莱特-费舍尔模型：具有等位基因分布 q 的群体产生具有等位基因分布 w 的后代，然后产生具有等位基因分布 p 的后代，依此类推。链式散度具有与 K-L 散度相同的一些属性（例如参数中的联合凸性），并且出现在与 K-L 散度相同的一些设置的 k 步版本中（例如信息投影和条件极限定理）。我们进一步描述了定义中出现的分布的最优 k 步“路径”，并将我们的发现应用于 Wright-Fisher 过程的大偏差分析。我们通过先前研究的连续极限与信息几何建立联系，其中步数趋于无穷大，极限路径是费舍尔信息度量中的测地线。最后，我们提供了链散度的热力学解释（作为适当定义的麦克斯韦妖的运行速率），并陈述了一些自然扩展和应用（k 步互信息和 k 步最大似然推理）。我们发布了用于计算我们研究的对象的代码。
We define and characterize the “chained” Kullback-Leibler divergence minw D(p‖w) + D(w‖q) minimized over all intermediate distributions w and the analogous k-fold chained K-L divergence min D(p‖wk−1) + … + D(w2‖w1) + D(w1‖q) minimized over the entire path (w1,…,wk−1). This quantity arises in a large deviations analysis of a Markov chain on the set of types – the Wright-Fisher model of neutral genetic drift: a population with allele distribution q produces offspring with allele distribution w, which then produce offspring with allele distribution p, and so on. The chained divergences enjoy some of the same properties as the K-L divergence (like joint convexity in the arguments) and appear in k-step versions of some of the same settings as the K-L divergence (like information projections and a conditional limit theorem). We further characterize the optimal k-step “path” of distributions appearing in the definition and apply our findings in a large deviations analysis of the Wright-Fisher process. We make a connection to information geometry via the previously studied continuum limit, where the number of steps tends to infinity, and the limiting path is a geodesic in the Fisher information metric. Finally, we offer a thermodynamic interpretation of the chained divergence (as the rate of operation of an appropriately defined Maxwell’s demon) and we state some natural extensions and applications (a k-step mutual information and k-step maximum likelihood inference). We release code for computing the objects we study.