StreaMRAK a streaming multi-resolution adaptive kernel algorithm
StreaMRAK a streaming multi-resolution adaptive kernel algorithm
复制标题
DOI：
10.1016/j.amc.2022.127112
复制
发表时间：
2021-08
期刊：
影响因子：
--
通讯作者：
Andreas Oslandsbotn;Ž. Kereta;Valeriya Naumova;Y. Freund;A. Cloninger
中科院分区：
文献类型：
--
作者：
Andreas Oslandsbotn;Ž. Kereta;Valeriya Naumova;Y. Freund;A. Cloninger
核岭回归（KRR）是非线性非参数学习的流行方案。然而，现有的KRR实现要求所有数据都存储在主存中，这严重限制了KRR在数据大小远远超过内存大小的情况下的使用。此类应用在数据挖掘、生物信息学和控制领域越来越常见。对于内存太大的数据集进行计算的一种强大范例是计算的流模型，在该模型中，我们一次处理一个数据样本，在继续处理下一个样本之前丢弃每个样本。在本文中，我们提出了 StreaMRAK - KRR 的流媒体版本。 StreaMRAK 通过将问题划分为多个分辨率级别来改进现有的 KRR 方案，从而可以不断改进预测。该算法通过持续有效地将新样本集成到训练模型中来减少内存需求。通过一种新颖的子采样方案，StreaMRAK 通过创建原始数据的 asketch 来降低内存和计算复杂性，其中子采样密度适应内核的带宽和数据的局部维度。我们提出了关于两个综合问题和双摆轨迹预测的展示研究。结果表明，该算法快速且准确。
Kernel ridge regression (KRR) is a popular scheme for non-linear non-parametric learning. However, existing implementations of KRR require that all the data is stored in the main memory, which severely limits the use of KRR in contexts where data size far exceeds the memory size. Such applications are increasingly common in data mining, bioinformatics, and control. A powerful paradigm for computing on data sets that are too large for memory is thestreaming model of computation, where we process one data sample at a time, discarding each sample before moving on to the next one. In this paper, we propose StreaMRAK - a streaming version of KRR. StreaMRAK improves on existing KRR schemes by dividing the problem into several levels of resolution, which allows continual refinement to the predictions. The algorithm reduces the memory requirement by continuously and efficiently integrating new samples into the training model. With a novel sub-sampling scheme, StreaMRAK reduces memory and computational complexities by creating asketchof the original data, where the sub-sampling density is adapted to the bandwidth of the kernel and the local dimensionality of the data. We present a showcase study on two synthetic problems and the prediction of the trajectory of a double pendulum. The results show that the proposed algorithm is fast and accurate.