Memory Efficient Meta-Learning with Large Images
Memory Efficient Meta-Learning with Large Images
复制标题
DOI：
--
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
J. Bronskill;Daniela Massiceti;Massimiliano Patacchiola;Katja Hofmann;Sebastian Nowozin;Richard E. Turner
中科院分区：
文献类型：
--
作者：
J. Bronskill;Daniela Massiceti;Massimiliano Patacchiola;Katja Hofmann;Sebastian Nowozin;Richard E. Turner
用于少镜头分类的元学习方法在测试时计算效率很高，只需要几个优化步骤或一次前向传递就可以学习新任务，但它们仍然需要高度的内存密集型训练。之所以会出现这种限制，是因为必须先处理任务的整个支持集(可包含多达1000个图像)，然后才能采取优化步骤。因此，要利用大图像提供的性能提升，需要在多个GPU上并行元学习器(这可能不可用)，或者在内存限制的情况下在任务和图像大小之间进行权衡。我们通过提出LITE来改进这两个选项，这是一种通用的、内存高效的插曲训练方案，可以在单个GPU上对由大图像组成的大型任务进行元训练。我们通过观察任务的梯度可以分解成任务的训练图像上的梯度和来实现这一点。这使我们能够对任务的整个训练集执行前向传递，但通过仅反向传播这些图像的随机子集来实现显著的内存节约，我们展示了这些图像是完全梯度的无偏近似。我们使用LITE来培训元学习者，并展示真实世界轨道基准和具有挑战性的VTAB+MD基准中相对于领先元学习者的4个部分中的3个的新的最先进的准确性。Lite还使元学习者能够与迁移学习方法竞争，但计算成本只有测试时间的一小部分，因此与最近的说法形成了反差，即迁移学习是少数几次分类所需的全部。
Meta learning approaches to few-shot classification are computationally efficient at test time, requiring just a few optimization steps or single forward pass to learn a new task, but they remain highly memory-intensive to train. This limitation arises because a task's entire support set, which can contain up to 1000 images, must be processed before an optimization step can be taken. Harnessing the performance gains offered by large images thus requires either parallelizing the meta-learner across multiple GPUs, which may not be available, or trade-offs between task and image size when memory constraints apply. We improve on both options by proposing LITE, a general and memory efficient episodic training scheme that enables meta-training on large tasks composed of large images on a single GPU. We achieve this by observing that the gradients for a task can be decomposed into a sum of gradients over the task's training images. This enables us to perform a forward pass on a task's entire training set but realize significant memory savings by back-propagating only a random subset of these images which we show is an unbiased approximation of the full gradient. We use LITE to train meta-learners and demonstrate new state-of-the-art accuracy on the real-world ORBIT benchmark and 3 of the 4 parts of the challenging VTAB+MD benchmark relative to leading meta-learners. LITE also enables meta-learners to be competitive with transfer learning approaches but at a fraction of the test-time computational cost, thus serving as a counterpoint to the recent narrative that transfer learning is all you need for few-shot classification.