Dimensionality reduction and information extraction for pattern recognition

降维本研究将降维方法作为模式识别的一种预处理方法。虽然PCA（主成分分析）是一种最流行的传统降维方法，但对于我们的目的来说，它有一个缺点，因为它不使用附加在训练样本上的类信息。另一方面，另一种流行的方法LDA（线性判别分析）使用类信息。但是，它根据类的数量对降维的数量有限制。这种限制可能导致过度减少。为了克服这些问题，我们考虑了基于约简数据中类别之间差异的降维方法作为约简优度的标准。首先，我们研究了一种基于Kullback-Leibler信息测量分布差异的方法。该方法利用类信息，对降维的数量没有任何限制。通过对基本任务的实验，我们发现该方法在线性不可分任务等方面，与PCA和LDA相比，降低了错误分类率。该方法存在两个问题：(1)使用了多维正态分布的拟合；(2)在优化过程中需要迭代计算。对于(2)，我们提出了一种方法，该方法对整个数据进行线性变换，只白化一个类，然后进行主成分分析。它不再需要迭代计算，但具有与前一种方法相似的性质。另一方面，对于(1)，我们利用类间虚势对方法进行了改进。与以往的低维降维方法相比，该方法实现了较低的分类错误率。对于多标签学习任务，我们考虑了两种信息表示方法：神经网络方法和条件分布方法，并研究了它们的行为。少

Dimensionality ReductionThe present study considers dimensionality reduction methods as a preprocessing of pattern recognition. Though PCA(principal component analysis) is a most popular conventional method of dimensionality reduction, it has a drawback for our purpose because it does not use class informations which are attatched to training samples. On the other hand, another popular method LDA(linear discriminant analysis) uses class information. However, it has a restriction on the number of reduced dimension according to the number of classes. This restriction can cause excessive reduction. In order to overcome these problems, we have considered dimensionality reduction methods based on the difference between classes in the reduced data as a criterion for goodness ofreductions.First, we have examined a method in which we measure the difference of distributions based on Kullback-Leibler information. This method uses class informations, and it does not have any restriction on the nu … More mber of reduced dimension. Through experiments with fimdamental tasks, we have observed that this method decreases the rate of wrong classifications compared with PCA and LDA for linear non-separatable tasks, and so on.Two problems of this methods are (1)it uses fitting of multidimensional normal distributions, and (2)it needs iterative calculations in optimization. As for (2), we have proposed a method in which a linear transformation, that whitens only one class, is applied to the whole data and then PCA is performed. It does not need iterative calculations any more, whereas it has similar property to the previous method. On the other hand, as for (1), we improved our method with a virtual potential between different classes. The last methods realizes lower rate of wrong classifications than the previous methods for dimensionality reduction to particularity low dimensions.Information RepresentationFor multi-labeled learning tasks, we have considered two ways of information representation, neural network method and and conditional distribution method, and their behaviors are invesitigated. Less