Estimating Squared-Loss Mutual Information for Independent Component Analysis
Estimating Squared-Loss Mutual Information for Independent Component Analysis
复制标题
DOI：
10.1007/978-3-642-00599-2_17
复制
发表时间：
2009-03
期刊：
影响因子：
--
通讯作者：
Taiji Suzuki;Masashi Sugiyama
中科院分区：
文献类型：
--
作者：
Taiji Suzuki;Masashi Sugiyama
准确评价随机变量之间的统计独立性是独立成分分析（伊卡）的一个重要组成部分。本文采用互信息的平方损失变量作为独立性测度，并给出了其估计方法。我们的基本思想是直接估计概率密度，而不经过密度估计，这样就避免了密度估计的困难。在这种密度比方法中，一个自然的交叉验证过程可用于模型选择。由于这一点，所有的调整参数，如核宽度或正则化参数，可以客观地优化。这在无监督学习问题（如伊卡）中是一个非常有用的属性。基于这种新的独立性测度，我们提出了一种新的伊卡算法--最小二乘独立分量分析（LICA）。
Accurately evaluating statistical independence among random variables is a key component of Independent Component Analysis (ICA). In this paper, we employ a squared-loss variant of mutual information as an independence measure and give its estimation method. Our basic idea is to estimate theratioof probability densities directly without going through density estimation, by which a hard task of density estimation can be avoided. In this density-ratio approach, a natural cross-validation procedure is available for model selection. Thanks to this, all tuning parameters such as the kernel width or the regularization parameter can be objectively optimized. This is a highly useful property in unsupervised learning problems such as ICA. Based on this novel independence measure, we develop a new ICA algorithm namedLeast-squares Independent Component Analysis(LICA).