Meta-learning for mixed linear regression
Meta-learning for mixed linear regression
复制标题
DOI：
--
复制
发表时间：
2020-02
期刊：
影响因子：
--
通讯作者：
Weihao Kong;Raghav Somani;Zhao Song;S. Kakade;Sewoong Oh
中科院分区：
文献类型：
--
作者：
Weihao Kong;Raghav Somani;Zhao Song;S. Kakade;Sewoong Oh
在现代监督学习中，有很多任务，但其中许多仅与少量标记数据有关。这些包括来自医疗图像处理和机器人相互作用的数据。即使每个任务都不能孤立地进行有意义的培训，但通过利用一些相似之处，人们试图从过去的经验中跨越的任务进行元学习。我们研究了一个基本的兴趣问题：何时可以为缺乏大数据的任务弥补少量数据吗？我们专注于一个规范的场景，其中每个任务都是从$ k $线性回归的混合物中汲取的，并确定了足够的条件以进行这种优雅的交换；与大数据任务可用时相似的示例总数类似。为此，我们介绍了一种新颖的光谱方法，并表明我们可以在$ \ tilde \ omega（k^{3/2}）的帮助下有效地利用小型数据任务（k^{3/2}）$中等数据任务，每个数据任务每个都带有$ \ tilde \ omega（ k^{1/2}）$示例。
In modern supervised learning, there are a large number of tasks, but many of them are associated with only a small amount of labeled data. These include data from medical image processing and robotic interaction. Even though each individual task cannot be meaningfully trained in isolation, one seeks to meta-learn across the tasks from past experiences by exploiting some similarities. We study a fundamental question of interest: When can abundant tasks with small data compensate for lack of tasks with big data? We focus on a canonical scenario where each task is drawn from a mixture of $k$ linear regressions, and identify sufficient conditions for such a graceful exchange to hold; The total number of examples necessary with only small data tasks scales similarly as when big data tasks are available. To this end, we introduce a novel spectral approach and show that we can efficiently utilize small data tasks with the help of $\tilde\Omega(k^{3/2})$ medium data tasks each with $\tilde\Omega(k^{1/2})$ examples.