Random Feature-based Online Multi-kernel Learning in Environments with Unknown Dynamics
Random Feature-based Online Multi-kernel Learning in Environments with Unknown Dynamics
复制标题
DOI：
--
复制
发表时间：
2017-12
期刊：
影响因子：
--
通讯作者：
Yanning Shen;Tianyi Chen;G. Giannakis
中科院分区：
文献类型：
--
作者：
Yanning Shen;Tianyi Chen;G. Giannakis
基于核的方法在各种非线性学习任务中表现出良好的性能。它们中的大多数依赖于预先选择的内核，其谨慎的选择假定特定于任务的先验信息。特别是当后者不可用时，多核学习由于其从指定的内核字典中选择内核的灵活性而受到欢迎。利用随机特征近似及其最近的正交性促进的变体，本贡献开发了一个可扩展的多核学习计划（称为Raker），以获得所寻求的非线性学习函数“在飞行中”，首先用于静态环境。为了进一步提高在动态环境中的性能，自适应多核学习计划（称为AdaRaker）的开发使用加权组合的建议，从分层集成的专家。权重不仅考虑了每个内核对学习的贡献，还考虑了未知的动态。性能分析方面的静态和动态的遗憾。AdaRaker能够在具有未知动态的环境中跟踪非线性学习函数，并具有分析性能保证。合成和真实的数据集进行测试，以展示新算法的有效性，以及它们的性能。
Kernel-based methods exhibit well-documented performance in various nonlinear learning tasks. Most of them rely on a preselected kernel, whose prudent choice presumes task-specific prior information. Especially when the latter is not available, multi-kernel learning has gained popularity thanks to its flexibility in choosing kernels from a prescribed kernel dictionary. Leveraging the random feature approximation and its recent orthogonality-promoting variant, the present contribution develops a scalable multi-kernel learning scheme (termed Raker) to obtain the sought nonlinear learning function `on the fly,' first for static environments. To further boost performance in dynamic environments, an adaptive multi-kernel learning scheme (termed AdaRaker) is developed using weighted combinations of advices from hierarchical ensembles of experts. The weights account not only for each kernel's contribution to the learning, but also for the unknown dynamics. Performance is analyzed in terms of both static and dynamic regrets. AdaRaker is uniquely capable of tracking nonlinear learning functions in environments with unknown dynamics, with analytic performance guarantees. Tests with synthetic and real datasets are carried out to showcase the effectiveness of the novel algorithms, and their performance.