Provable Guarantees for Gradient-Based Meta-Learning
Provable Guarantees for Gradient-Based Meta-Learning
复制标题
DOI：
--
复制
发表时间：
2019-02
期刊：
影响因子：
--
通讯作者：
M. Khodak;Maria-Florina Balcan;Ameet Talwalkar
中科院分区：
文献类型：
--
作者：
M. Khodak;Maria-Florina Balcan;Ameet Talwalkar
我们通过在线凸优化的视角研究元学习问题，开发了一种元算法，弥合了流行的基于梯度的元学习和经典的基于正则化的多任务迁移方法之间的差距。我们的方法是第一个同时满足凸设置中良好的样本效率保证的方法，其泛化界限随着任务相似性的提高而提高，同时也可以在计算上扩展到现代深度学习架构和多任务设置。尽管它很简单，但在自然任务相似性假设下，该算法匹配任何此类参数传递方法的性能的下界，直到一个常数因子。我们在凸学习和深度学习设置中使用实验来验证和展示我们理论的适用性。
We study the problem of meta-learning through the lens of online convex optimization, developing a meta-algorithm bridging the gap between popular gradient-based meta-learning and classical regularization-based multi-task transfer methods. Our method is the first to simultaneously satisfy good sample efficiency guarantees in the convex setting, with generalization bounds that improve with task-similarity, while also being computationally scalable to modern deep learning architectures and the many-task setting. Despite its simplicity, the algorithm matches, up to a constant factor, a lower bound on the performance of any such parameter-transfer method under natural task similarity assumptions. We use experiments in both convex and deep learning settings to verify and demonstrate the applicability of our theory.