A Local Characterization for Wyner Common Information
A Local Characterization for Wyner Common Information
复制标题
DOI：
10.1109/isit44484.2020.9174206
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Shao-Lun Huang;Xiangxiang Xu;Lizhong Zheng;G. Wornell
中科院分区：
文献类型：
--
作者：
Shao-Lun Huang;Xiangxiang Xu;Lizhong Zheng;G. Wornell
hirschfeld - gebelein - rnyi （HGR）最大相关法与Wyner共同信息法具有相似的信息处理目的，即提取随机变量之间的共同知识结构，但这两种方法之间的关系通常不明确。在本文中，我们通过考虑弱依赖区域中的Wyner公共信息（称为ϵ-common信息）来证明这种关系。我们证明了HGR最大相关函数与ϵ-common信息中辅助随机变量估计的相对似然函数重合，从而建立了这些方法的基本联系。此外，我们将ϵ-common信息扩展到多个随机变量，并推导了一种基于数据变量共同信息提取特征函数的新算法。我们的方法通过MNIST问题得到了验证，并且可能在多模态数据分析中有用。
While the Hirschfeld-Gebelein-Rényi (HGR) maximal correlation and the Wyner common information share similar information processing purposes of extracting common knowledge structures between random variables, the relationships between these approaches are generally unclear. In this paper, we demonstrate such relationships by considering the Wyner common information in the weakly dependent regime, called ϵ-common information. We show that the HGR maximal correlation functions coincide with the relative likelihood functions of estimating the auxiliary random variables in ϵ-common information, which establishes the fundamental connections these approaches. Moreover, we extend the ϵ-common information to multiple random variables, and derive a novel algorithm for extracting feature functions of data variables regarding their common information. Our approach is validated by the MNIST problem, and can potentially be useful in multi-modal data analyses.