Why Does a Hilbertian Metric Work Efficiently in Online Learning With Kernels?
Why Does a Hilbertian Metric Work Efficiently in Online Learning With Kernels?
复制标题
DOI：
10.1109/lsp.2016.2598615
复制
发表时间：
2016-10
影响因子：
3.9
通讯作者：
M. Yukawa;K. Müller
中科院分区：
文献类型：
--
作者：
M. Yukawa;K. Müller
核化输入向量的自相关矩阵由平方Gram矩阵很好地近似（按字典大小缩小）。在特征空间中的输入协方差矩阵由基于字典元素的样本估计来逼近的条件下，这一点仍然成立，从而获得了对使用内核进行在线学习的一些基本见解。首先，与超平面投影沿着仿射子空间算法相关的自相关矩阵的特征值扩展近似为核归一化最小均方算法的特征值扩展的平方根。这澄清了由于使用希尔伯特度量而导致快速收敛背后的机制。其次，为了有效的函数估计，通常需要通过考虑输入向量的分布来构造字典，以满足条件。计算机实验验证了理论分析的正确性。
The autocorrelation matrix of the kernelized input vector is well approximated by the squared Gram matrix (scaled down by the dictionary size). This holds true under the condition that the input covariance matrix in the feature space is approximated by its sample estimate based on the dictionary elements, leading to a couple of fundamental insights into online learning with kernels. First, the eigenvalue spread of the autocorrelation matrix relevant to the hyperplane projection along affine subspace algorithm is approximately a square root of that for the kernel normalized least mean square algorithm. This clarifies the mechanism behind fast convergence due to the use of a Hilbertian metric. Second, for efficient function estimation, the dictionary needs to be constructed in general by taking into account the distribution of the input vector, so as to satisfy the condition. The theoretical results are justified by computer experiments.