Learning linear PCA with convex semi-definite programming
Learning linear PCA with convex semi-definite programming
复制标题
DOI：
10.1016/j.patcog.2007.01.022
复制
发表时间：
2007-10
期刊：
影响因子：
--
通讯作者：
Qing Tao;Gao-wei Wu;Jue Wang
中科院分区：
文献类型：
--
作者：
Qing Tao;Gao-wei Wu;Jue Wang
本文的目的是利用支持向量机 (SVM) 的性质来学习线性主成分。为此，构建了一个用于确定投影方向的完整的类似SVM的线性PCA（SVPCA）框架，其中引入了新的预期风险和保证金。在此框架内，提出了一个用于最大化裕度的新半定规划问题，并建立了支持向量的新定义。作为常规 PCA 的加权情况，如果所有样本在数据压缩中发挥相同的作用，则我们的 SVPCA 与常规 PCA 一致。理论解释表明SVPCA基于基于边际的泛化界限，从而保证了良好的预测能力。此外，使用 SVM 中的软思想实现了具有可解释参数的 SVPCA 的鲁棒形式。最大的优点在于，由于半定优化问题的凸性，SVPCA是一种没有局部最小值的学习算法。为了验证 SVPCA 的性能，进行了多次实验，数值结果表明其泛化能力优于常规 PCA。最后还对存在的一些问题进行了讨论。
The aim of this paper is to learn a linear principal component using the nature of support vector machines (SVMs). To this end, a complete SVM-like framework of linear PCA (SVPCA) for deciding the projection direction is constructed, where new expected risk and margin are introduced. Within this framework, a new semi-definite programming problem for maximizing the margin is formulated and a new definition of support vectors is established. As a weighted case of regular PCA, our SVPCA coincides with the regular PCA if all the samples play the same part in data compression. Theoretical explanation indicates that SVPCA is based on a margin-based generalization bound and thus good prediction ability is ensured. Furthermore, the robust form of SVPCA with a interpretable parameter is achieved using the soft idea in SVMs. The great advantage lies in the fact that SVPCA is a learning algorithm without local minima because of the convexity of the semi-definite optimization problems. To validate the performance of SVPCA, several experiments are conducted and numerical results have demonstrated that their generalization ability is better than that of regular PCA. Finally, some existing problems are also discussed.