Towards a Unified Quadrature Framework for Large-Scale Kernel Machines
Towards a Unified Quadrature Framework for Large-Scale Kernel Machines
复制标题
DOI：
10.1109/tpami.2021.3120183
复制
发表时间：
2020-11
影响因子：
23.6
通讯作者：
Fanghui Liu;Xiaolin Huang;Yudong Chen;J. Suykens
中科院分区：
文献类型：
--
作者：
Fanghui Liu;Xiaolin Huang;Yudong Chen;J. Suykens
在本文中，我们开发了一个正交框架的大规模内核机通过数值积分表示。考虑到典型核的积分域和测度，例如，高斯核，反余弦核，是完全对称的，我们利用确定性的完全对称插值规则，有效地计算正交节点和相关的权重核近似。所开发的插值规则能够减少所需节点的数量，同时保持较高的逼近精度。进一步，我们利用经典的蒙特-卡罗采样和控制变量技术对上述确定性规则进行随机化，其优点有两个：1）所提出的随机规则使得特征映射的维数灵活变化，这样我们就可以通过调节维数来控制原始核和近似核之间的差异。2)我们的随机规则具有良好的统计特性，无偏性和方差减少，收敛速度快。此外，我们阐明了我们的确定性/随机插值规则和当前的核近似求积规则之间的关系，包括稀疏网格求积和随机球径向规则，从而统一这些方法在我们的框架下。在几个基准数据集上的实验结果表明，我们的方法与其他代表性的基于核近似的方法相比毫不逊色。
In this paper, we develop a quadrature framework for large-scale kernel machines via a numerical integration representation. Considering that the integration domain and measure of typical kernels, e.g., Gaussian kernels, arc-cosine kernels, are fully symmetric, we leverage deterministic fully symmetric interpolatory rules to efficiently compute quadrature nodes and associated weights for kernel approximation. The developed interpolatory rules are able to reduce the number of needed nodes while retaining a high approximation accuracy. Further, we randomize the above deterministic rules by the classical Monte-Carlo sampling and control variates techniques with two merits: 1) The proposed stochastic rules make the dimension of the feature mapping flexibly varying, such that we can control the discrepancy between the original and approximate kernels by tuning the dimnension. 2) Our stochastic rules have nice statistical properties of unbiasedness and variance reduction with fast convergence rate. In addition, we elucidate the relationship between our deterministic/stochastic interpolatory rules and current quadrature rules for kernel approximation, including the sparse grids quadrature and stochastic spherical-radial rules, thereby unifying these methods under our framework. Experimental results on several benchmark datasets show that our methods compare favorably with other representative kernel approximation based methods.