LSMI-Sinkhorn: Semi-supervised Mutual Information Estimation with Optimal Transport
LSMI-Sinkhorn: Semi-supervised Mutual Information Estimation with Optimal Transport
复制标题
DOI：
10.1007/978-3-030-86486-6_40
复制
发表时间：
2019-09
期刊：
影响因子：
--
通讯作者：
Yanbin Liu;M. Yamada;Yao-Hung Hubert Tsai;Tam Le;R. Salakhutdinov;Yi Yang
中科院分区：
文献类型：
--
作者：
Yanbin Liu;M. Yamada;Yao-Hung Hubert Tsai;Tam Le;R. Salakhutdinov;Yi Yang
互信息估计是一个重要的统计和机器学习问题。为了从数据中估计互信息，通常的做法是准备一组配对样本。然而，在许多情况下，很难获得大量的数据对。为了解决这个问题，我们提出了半监督平方损失互信息（SMI）估计方法，使用少量的配对样本和可用的非配对样本。我们首先通过密度比函数表示SMI，其中期望由来自边缘的样本及其分配参数近似。目标是制定使用最优运输问题和二次规划。然后，我们介绍了最小二乘互信息与Sinkhorn（LSMI-Sinkhorn）算法的有效优化。通过实验，我们首先证明，该方法可以估计SMI没有大量的配对样本。然后，我们展示了所提出的LSMI-Sinkhorn算法在各种类型的机器学习问题（如图像匹配和相册摘要）上的有效性。代码可以在 https://github.com/csyanbin/LSMI-Sinkhorn .
Estimating mutual information is an important statistics and machine learning problem. To estimate the mutual information from data, a common practice is preparing a set of paired samples. However, in many situations, it is difficult to obtain a large number of data pairs. To address this problem, we propose the semi-supervised Squared-loss Mutual Information (SMI) estimation method using a small number of paired samples and the available unpaired ones. We first represent SMI through the density ratio function, where the expectation is approximated by the samples from marginals and its assignment parameters. The objective is formulated using the optimal transport problem and quadratic programming. Then, we introduce theLeast-SquaresMutualInformation withSinkhorn(LSMI-Sinkhorn) algorithm for efficient optimization. Through experiments, we first demonstrate that the proposed method can estimate the SMI without a large number of paired samples. Then, we show the effectiveness of the proposed LSMI-Sinkhorn algorithm on various types of machine learning problems such as image matching and photo album summarization. Code can be found at https://github.com/csyanbin/LSMI-Sinkhorn .