Variational Hyperparameter Inference for Few-Shot Learning Across Domains
Variational Hyperparameter Inference for Few-Shot Learning Across Domains
复制标题
跨域少发学习的变分超参数推理
DOI：
10.1109/tcsvt.2022.3188462
复制
发表时间：
2022-11
影响因子：
8.4
通讯作者：
Lei Zhang;Liyun Zuo;Baoyan Wang;Xin Li;Xiantong Zhen
中科院分区：
文献类型：
--
作者：
Lei Zhang;Liyun Zuo;Baoyan Wang;Xin Li;Xiantong Zhen
最近，少数人学习研究的焦点一直集中在元学习的发展上，即元学习者接受关于各种任务的培训，希望能够概括到新的任务。元训练和元测试中的任务通常被假设来自同一个领域，这在现实世界的场景中并不一定成立。在本文中，我们提出了跨域少射学习的变分超参数推理。该变分超参数推理算法在模型不可知元学习算法的基础上，将元学习和变分推理结合到超参数优化中，使元学习器具有跨域泛化的适应性。特别是，我们选择学习包括学习率和权值衰减在内的自适应超参数，以避免在跨域的少量标注样本面前失败。此外，我们将超参数建模为分布而不是固定值，这将通过捕捉不确定性来进一步增强泛化能力。在两个基准数据集上进行了大量的实验，包括域内和跨域的少镜头学习数据集。结果表明，我们的方法始终优于以前的方法，综合消融研究进一步验证了它在域内和域间的少镜头学习的有效性。
The focus of few shot learning research has been on the development of meta-learning recently, where a meta-learner is trained on a variety of tasks in hopes of being generalizable to new tasks. Tasks in meta training and meta test are usually assumed to be from the same domain, which would not necessarily hold in real world scenarios. In this paper, we propose variational hyperparameter inference for few-shot learning across domains. Based on an especially successful algorithm named model agnostic meta learning, the proposed variational hyperparameter inference integrates meta learning and variational inference into the optimization of hyperparameters, which enables the meta-learner with adaptivity for generalization across domains. In particular, we choose to learn adaptive hyperparameters including the learning rate and weight decay to avoid the failure in the face of few labeled examples across domain. Moreover, we model hyperparameters as distributions instead of fixed values, which will further enhance the generalization ability by capturing the uncertainty. Extensive experiments are conducted on two benchmark datasets including few shot learning dataset within-domain and across-domain. The results demonstrate that our methods outperforms previous approaches consistently, and comprehensive ablation studies further validate its effectiveness on few shot learning both within domains and across domains.