Sparse supervised principal component analysis (SSPCA) for dimension reduction and variable selection
Sparse supervised principal component analysis (SSPCA) for dimension reduction and variable selection
复制标题
DOI：
10.1016/j.engappai.2017.07.004
复制
发表时间：
2017-10-01
影响因子：
8
通讯作者：
Ersboll, Bjarne K.
中科院分区：
文献类型：
--
作者：
Sharifzadeh, Sara;Ghodsi, Ali;Ersboll, Bjarne K.
主成分分析（PCA）（1）是一种主要的无监督降维预处理方法。当训练标签可用时，值得使用监督PCA策略。在需要降维和变量选择的情况下，稀疏PCA（SPCA）方法是优选的。本文提出了一种稀疏监督主成分分析（SSPCA）的预处理方法。这种方法特别适用于高维输入需要使用稀疏方法的问题，并且目标标签也可用于指导变量选择策略。这种方法在许多工程和科学问题中是有价值的，当训练样本的数量也有限。希尔伯特施密特独立准则（HSIC）是用来形成一个目标的损失函数的最小化的基础上，和L-1范数用于正则化的特征向量。虽然所提出的目标函数允许输入和响应矩阵之间的线性和非线性关系的稀疏低秩解决方案，但在这种情况下，其他类似的方法仅基于线性模型。基于惩罚矩阵分解（PMD）算法求解该目标。我们比较了该方法与PCA，PMD为基础的SPCA和监督PCA。此外，由于SSPCA与稀疏偏最小二乘（SPLS）目标函数的相似性，本文还将SSPCA与SPLS进行了比较。模拟和真实的数据集的实验结果表明，SSPCA提供了一个适当的权衡精度和稀疏性。比较表明，在稀疏性方面，SSPCA执行最高级别的变量减少，而且，在精度方面，它是最成功的方法之一。因此，SSPCA得到的特征向量可用于各种高维问题的特征选择。(C)2017爱思唯尔有限公司版权所有
Principal component analysis (PCA)(1) is one of the main unsupervised pre-processing methods for dimension reduction. When the training labels are available, it is worth using a supervised PCA strategy. In cases that both dimension reduction and variable selection are required, sparse PCA (SPCA) methods are preferred. In this paper, a sparse supervised PCA (SSPCA) method is proposed for pre-processing. This method is appropriate especially in problems where, a high dimensional input necessitates the use of a sparse method and a target label is also available to guide the variable selection strategy. Such a method is valuable in many Engineering and scientific problems, when the number of training samples is also limited. The Hilbert Schmidt Independence Criteria (HSIC) is used to form an objective based on minimization of a loss function and an L-1 norm is used for regularization of the Eigen vectors. While the proposed objective function allows a sparse low rank solution for both linear and non-linear relationships between the input and response matrices, other similar methods in this case are only based on a linear model. The objective is solved based on penalized matrix decomposition (PMD) algorithm. We compare the proposed method with PCA, PMD-based SPCA and supervised PCA. In addition, SSPCA is also compared with sparse partial least squares (SPLS), due to the similarity between the two objective functions. Experimental results from the simulated as well as real data sets show that, SSPCA provides an appropriate trade-off between accuracy and sparsity. Comparisons show that, in terms of sparsity, SSPCA performs the highest level of variable reduction and also, in terms of accuracy it is one of the most successful methods. Therefore, the Eigen vectors found by SSPCA can be used for feature selection in various high dimensional problems. (C) 2017 Elsevier Ltd. All rights reserved.