Approximating Mutual Information by Maximum Likelihood Density Ratio Estimation
Approximating Mutual Information by Maximum Likelihood Density Ratio Estimation
复制标题
DOI：
--
复制
发表时间：
2008-09
期刊：
影响因子：
--
通讯作者：
Taiji Suzuki;Masashi Sugiyama;J. Sese;T. Kanamori
中科院分区：
文献类型：
--
作者：
Taiji Suzuki;Masashi Sugiyama;J. Sese;T. Kanamori
互信息在各种数据处理任务中非常有用，例如特征选择或独立成分分析。在本文中，我们提出了一种基于密度比函数的最大似然估计来近似互信息的新方法。我们的方法称为最大似然互信息（MLMI），具有几个有吸引力的特性，例如，不涉及密度估计，它是单次过程，可以有效地计算全局最优解，并且交叉验证可用于模型选择。数值实验表明，MLMI 与现有方法相比具有优势。
Mutual information is useful in various data processing tasks such as feature selection or independent component analysis. In this paper, we propose a new method of approximating mutual information based on maximum likelihood estimation of a density ratio function. Our method, called Maximum Likelihood Mutual Information (MLMI), has several attractive properties, e.g., density estimation is not involved, it is a single-shot procedure, the global optimal solution can be efficiently computed, and cross-validation is available for model selection. Numerical experiments show that MLMI compares favorably with existing methods.