Learning To Learn Around A Common Mean
Learning To Learn Around A Common Mean
复制标题
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Giulia Denevi;C. Ciliberto;Dimitris Stamos;M. Pontil
中科院分区：
文献类型：
--
作者：
Giulia Denevi;C. Ciliberto;Dimitris Stamos;M. Pontil
学习到学习（LTL）或元学习的问题越来越受到关注，因为最近的经验证据表明其在应用中的有效性。LTL的目标是选择一种算法，该算法可以很好地处理从元分布中采样的任务。在这项工作中，我们考虑了由岭回归的一个变体给出的算法族，其中正则化子是到未知均值向量的平方距离。我们证明了，在这种情况下，LTL问题可以转化为一个最小二乘（LS）问题，我们利用一种新的Meta算法来有效地解决它。在每次迭代中，元算法只处理一个数据集。具体来说，它首先估计随机LS目标函数，通过将该数据集分成两个子集，分别用于训练和测试内部算法。其次，它执行一个随机梯度步骤的估计值。在特定的假设下，我们提出了我们的元算法的泛化误差，这表明正确的分裂参数的选择。当问题的超参数固定时，即使样本大小保持不变，随着任务数量的增加，这个界限也是一致的。初步实验证实了我们的理论研究结果，突出了我们的方法的优势，相对于独立的任务学习。
The problem of learning-to-learn (LTL) or meta-learning is gaining increasing attention due to recent empirical evidence of its effectiveness in applications. The goal addressed in LTL is to select an algorithm that works well on tasks sampled from a meta-distribution. In this work, we consider the family of algorithms given by a variant of Ridge Regression, in which the regularizer is the square distance to an unknown mean vector. We show that, in this setting, the LTL problem can be reformulated as a Least Squares (LS) problem and we exploit a novel meta- algorithm to efficiently solve it. At each iteration the meta-algorithm processes only one dataset. Specifically, it firstly estimates the stochastic LS objective function, by splitting this dataset into two subsets used to train and test the inner algorithm, respectively. Secondly, it performs a stochastic gradient step with the estimated value. Under specific assumptions, we present a bound for the generalization error of our meta-algorithm, which suggests the right splitting parameter to choose. When the hyper-parameters of the problem are fixed, this bound is consistent as the number of tasks grows, even if the sample size is kept constant. Preliminary experiments confirm our theoretical findings, highlighting the advantage of our approach, with respect to independent task learning.