Feature Extraction With Deep Neural Networks by a Generalized Discriminant Analysis
Feature Extraction With Deep Neural Networks by a Generalized Discriminant Analysis
复制标题
DOI：
10.1109/tnnls.2012.2183645
复制
发表时间：
2012-02
影响因子：
10.4
通讯作者：
A. Stuhlsatz;J. Lippel;Thomas Zielke
中科院分区：
文献类型：
--
作者：
A. Stuhlsatz;J. Lippel;Thomas Zielke
我们提出了一种特征提取方法，该方法是基于深度神经网络（DNN）的经典线性判别分析（LDA）的推广。对于LDA，假设从独立的高斯类条件生成的判别特征。该模型的优点是特征空间的内在维数受类别数的限制，最佳判别函数是线性的。不幸的是，线性变换不足以从任意分布的原始测量中提取最佳判别特征。本文提出的广义判别分析（GerDA）使用DNN以半监督方式学习的非线性变换。我们表明，基于我们的方法的特征提取在现实世界的识别和检测任务，如手写数字识别和人脸检测显示出优异的性能。在一系列的实验中，我们评估GerDA功能方面的降维，可视化，分类和检测。此外，我们表明，GerDA DNN可以将真正的高维输入数据预处理为低维表示，即使使用简单的线性预测器或相似性度量，也可以进行准确的预测。
We present an approach to feature extraction that is a generalization of the classical linear discriminant analysis (LDA) on the basis of deep neural networks (DNNs). As for LDA, discriminative features generated from independent Gaussian class conditionals are assumed. This modeling has the advantages that the intrinsic dimensionality of the feature space is bounded by the number of classes and that the optimal discriminant function is linear. Unfortunately, linear transformations are insufficient to extract optimal discriminative features from arbitrarily distributed raw measurements. The generalized discriminant analysis (GerDA) proposed in this paper uses nonlinear transformations that are learnt by DNNs in a semisupervised fashion. We show that the feature extraction based on our approach displays excellent performance on real-world recognition and detection tasks, such as handwritten digit recognition and face detection. In a series of experiments, we evaluate GerDA features with respect to dimensionality reduction, visualization, classification, and detection. Moreover, we show that GerDA DNNs can preprocess truly high-dimensional input data to low-dimensional representations that facilitate accurate predictions even if simple linear predictors or measures of similarity are used.