MAML is a Noisy Contrastive Learner
MAML is a Noisy Contrastive Learner
复制标题
MAML 是一个嘈杂的对比学习器
DOI：
--
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Pin
中科院分区：
文献类型：
--
作者：
Chia;Wei;Pin
模型不可知元学习（MAML）是目前最流行和广泛采用的元学习算法之一，它在各种学习问题上取得了显著的成功。然而，由于MAML的独特设计，即嵌套的内循环和外循环更新分别控制着任务特定的学习和元模型中心的学习，MAML的潜在学习目标仍然是隐含的，因此阻碍了对它的更直接的理解。在本文中，我们对MAML的工作机制提供了一个新的视角，并发现：MAML类似于使用监督对比目标函数的元学习器，其中查询特征被拉向同一类的支持特征，并反对不同类的支持特征，其中这种对比性通过基于余弦相似性的分析实验性地验证艾德。此外，我们的分析表明，香草MAML算法有一个不希望的干扰项来自随机初始化和跨任务的相互作用。因此，我们提出了一种简单但有效的技术，归零技巧，以减轻这种干扰，其中广泛的实验，然后进行miniImagenet和Omniglot数据集，以证明我们提出的技术带来的一致的改善，从而验证其有效性。
Model-agnostic meta-learning (MAML) is one of the most popular and widely-adopted meta-learning algorithms nowadays, which achieves remarkable success in various learning problems. Yet, with the unique design of nested inner-loop and outer-loop updates which respectively govern the task-specific and meta-model-centric learning, the underlying learning objective of MAML still remains implicit and thus impedes a more straightforward understanding of it. In this paper, we provide a new perspective to the working mechanism of MAML and discover that: MAML is analogous to a meta-learner using a supervised contrastive objective function, where the query features are pulled towards the support features of the same class and against those of different classes, in which such contrastiveness is experimentally verified via an analysis based on the cosine similarity. Moreover, our analysis reveals that the vanilla MAML algorithm has an undesirable interference term originating from the random initialization and the cross-task interaction. We therefore propose a simple but effective technique, zeroing trick, to alleviate such interference, where the extensive experiments are then conducted on both miniImagenet and Omniglot datasets to demonstrate the consistent improvement brought by our proposed technique thus validating its effectiveness.
DOI：
--
发表时间：
2019-09
期刊：
--
影响因子：
--
作者：
A. Rajeswaran;Chelsea Finn;S. Kakade;S. Levine
通讯作者：
A. Rajeswaran;Chelsea Finn;S. Kakade;S. Levine