A Conditional Entropy Minimization Criterion for Dimensionality Reduction and Multiple Kernel Learning
A Conditional Entropy Minimization Criterion for Dimensionality Reduction and Multiple Kernel Learning
复制标题
DOI：
10.1162/neco_a_00027
复制
发表时间：
2010-11
影响因子：
2.9
通讯作者：
H. Hino;Noboru Murata
中科院分区：
文献类型：
--
作者：
H. Hino;Noboru Murata
如何在不损失数据本质信息的前提下对高维数据进行降维是信息处理的一个重要任务。当训练数据的类别标签可用时，Fisher判别分析（FDA）已被广泛使用。然而，FDA的最优性仅在非常受限的理想情况下才能得到保证，并且经常观察到FDA并不能为许多真实的问题提供良好的分类表面。本文从信息论的角度出发，研究了有监督的降维问题，提出了一种基于类条件熵最小化的降维框架。所提出的线性降维技术的理论和实验验证。然后，通过核Fisher判别分析（KFDA），在该框架下处理多核学习问题，提出了一种迭代优化分类函数参数和核组合系数的新算法。实验表明，该算法在大规模基准数据集上与KFDA相当或优于KFDA，并且在酵母蛋白质功能注释任务上与其他多核学习技术相当。
Reducing the dimensionality of high-dimensional data without losing its essential information is an important task in information processing. When class labels of training data are available, Fisher discriminant analysis (FDA) has been widely used. However, the optimality of FDA is guaranteed only in a very restricted ideal circumstance, and it is often observed that FDA does not provide a good classification surface for many real problems. This letter treats the problem of supervised dimensionality reduction from the viewpoint of information theory and proposes a framework of dimensionality reduction based on class-conditional entropy minimization. The proposed linear dimensionality-reduction technique is validated both theoretically and experimentally. Then, through kernel Fisher discriminant analysis (KFDA), the multiple kernel learning problem is treated in the proposed framework, and a novel algorithm, which iteratively optimizes the parameters of the classification function and kernel combination coefficients, is proposed. The algorithm is experimentally shown to be comparable to or outperforms KFDA for large-scale benchmark data sets, and comparable to other multiple kernel learning techniques on the yeast protein function annotation task.