Online-Within-Online Meta-Learning
Online-Within-Online Meta-Learning
复制标题
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Giulia Denevi;Dimitris Stamos;Carlo Ciliberto;Massimiliano Pontil
中科院分区：
文献类型：
--
作者：
Giulia Denevi;Dimitris Stamos;Carlo Ciliberto;Massimiliano Pontil
我们研究了在完全在线的元学习环境中学习一系列任务的问题。我们的目标是利用任务之间的相似性，逐步适应内部的在线算法，以引起低平均累积误差的任务。我们专注于一个家庭的内部算法的基础上的参数化的变体在线镜像下降。内部算法通过在线镜像下降元算法使用相应的任务内最小正则化经验风险作为元损失进行增量调整。为了保持过程完全在线，我们近似的元次梯度的在线内部算法。近似误差的上限允许我们推导出所提出的方法的累积误差界。我们的分析也可以通过online-to-batch参数转换为统计设置。我们实例化的框架中的元参数是一个共同的偏置向量或特征映射的两个例子。最后，初步的数值实验证实了我们的理论研究结果。
We study the problem of learning a series of tasks in a fully online Meta-Learning setting. The goal is to exploit similarities among the tasks to incrementally adapt an inner online algorithm in order to incur a low averaged cumulative error over the tasks. We focus on a family of inner algorithms based on a parametrized variant of online Mirror Descent. The inner algorithm is incrementally adapted by an online Mirror Descent meta-algorithm using the corresponding within-task minimum regularized empirical risk as the meta-loss. In order to keep the process fully online, we approximate the meta-subgradients by the online inner algorithm. An upper bound on the approximation error allows us to derive a cumulative error bound for the proposed method. Our analysis can also be converted to the statistical setting by online-to-batch arguments. We instantiate two examples of the framework in which the meta-parameter is either a common bias vector or feature map. Finally, preliminary numerical experiments confirm our theoretical findings.