Generating feature spaces for linear algorithms with regularized sparse kernel slow feature analysis
Generating feature spaces for linear algorithms with regularized sparse kernel slow feature analysis
复制标题
DOI：
10.1007/s10994-012-5300-0
复制
发表时间：
2012-06
期刊：
影响因子：
7.5
通讯作者：
Wendelin Böhmer;S. Grünewälder;H. Nickisch;K. Obermayer
中科院分区：
文献类型：
--
作者：
Wendelin Böhmer;S. Grünewälder;H. Nickisch;K. Obermayer
如果没有非线性的基函数，很多问题都不能用线性算法来解决。本文提出了一种利用慢特征分析(SFA)自动构造基函数的方法。这种无监督学习方法的非线性优化在给定时间序列的未知潜在空间上产生一个正交基。因此，与主成分分析等方法相比，SFA非常适合于直接利用潜在空间的技术。现实世界的时间序列可能很复杂，当前的SFA算法要么功能不够强大，要么倾向于过度适应。我们利用核Trickin和稀疏化相结合的方法开发了一种核化的SFA算法，它为大数据集提供了一个强大的函数类。稀疏性是通过一种新颖的匹配追逐方法实现的，这种方法也可以应用于其他任务。然而，对于较小的数据集，核SFA方法会导致过拟合和数值不稳定。为了执行稳定的解决方案，我们在SFA目标中引入了监管。我们假设我们的算法生成一个特征空间，该特征空间类似于给定真实世界时间序列背后的潜在变量的未知空间中的傅立叶基。我们以元音分类任务为例，与分析核主成分分析相比较，对这一假设进行了评估。实验结果表明，核SFA比核主成分分析具有更高的分类精度，在对潜在变量进行编码方面具有明显的优势。
Without non-linear basis functions many problems can not be solved by linear algorithms. This article proposes a method to automatically construct such basis functions withslow feature analysis(SFA). Non-linear optimization of this unsupervised learning method generates an orthogonal basis on the unknown latent space for a given time series. In contrast to methods like PCA, SFA is thus well suited for techniques that make direct use of the latent space. Real-world time series can be complex, and current SFA algorithms are either not powerful enough or tend to over-fit. We make use of thekernel trickin combination withsparsificationto develop a kernelized SFA algorithm which provides a powerful function class for large data sets. Sparsity is achieved by a novelmatching pursuitapproach that can be applied to other tasks as well. For small data sets, however, the kernel SFA approach leads to over-fitting and numerical instabilities. To enforce a stable solution, we introduceregularizationto the SFA objective. We hypothesize thatour algorithm generates a feature space that resembles a Fourier basis in the unknown space of latent variables underlying a given real-world time series. We evaluate this hypothesis at the example of avowel classificationtask in comparison tosparse kernel PCA. Our results show excellent classification accuracy and demonstrate the superiority of kernel SFA over kernel PCA in encoding latent variables.