FiT: Parameter Efficient Few-shot Transfer Learning for Personalized and Federated Image Classification
FiT: Parameter Efficient Few-shot Transfer Learning for Personalized and Federated Image Classification
复制标题
DOI：
10.48550/arxiv.2206.08671
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Aliaksandra Shysheya;J. Bronskill;Massimiliano Patacchiola;Sebastian Nowozin;Richard E. Turner
中科院分区：
文献类型：
--
作者：
Aliaksandra Shysheya;J. Bronskill;Massimiliano Patacchiola;Sebastian Nowozin;Richard E. Turner
现代深度学习系统越来越多地被部署在个性化和联合学习等情况下，其中需要支持i)基于少量数据的学习，以及ii)通信高效的分布式训练协议。在这项工作中，我们开发了电影传输(FIT)，它通过结合迁移学习(固定的预先训练的主干和微调的电影适配器层)和元学习(自动配置朴素贝叶斯分类器和情节训练)的思想来满足图像分类设置的这些要求，从而在低镜头下产生参数高效的模型，并且具有较高的分类精度。由此产生的参数效率对于实现少量学习、用于个性化的廉价模型更新以及通信高效的联合学习是关键。我们在广泛的下游数据集上进行了FIT实验，结果表明，该算法在低发送率下获得了比领先的Big Transfer(BIT)算法更好的分类精度，并且在具有挑战性的VTAB-1k基准测试中达到了最高的精度，使用了不到1%的可更新参数。最后，我们展示了FIT在分布式低速应用中的参数效率和卓越的精度，包括模型个性化和联邦学习，其中模型更新量是一个重要的性能指标。
Modern deep learning systems are increasingly deployed in situations such as personalization and federated learning where it is necessary to support i) learning on small amounts of data, and ii) communication efficient distributed training protocols. In this work, we develop FiLM Transfer (FiT) which fulfills these requirements in the image classification setting by combining ideas from transfer learning (fixed pretrained backbones and fine-tuned FiLM adapter layers) and meta-learning (automatically configured Naive Bayes classifiers and episodic training) to yield parameter efficient models with superior classification accuracy at low-shot. The resulting parameter efficiency is key for enabling few-shot learning, inexpensive model updates for personalization, and communication efficient federated learning. We experiment with FiT on a wide range of downstream datasets and show that it achieves better classification accuracy than the leading Big Transfer (BiT) algorithm at low-shot and achieves state-of-the art accuracy on the challenging VTAB-1k benchmark, with fewer than 1% of the updateable parameters. Finally, we demonstrate the parameter efficiency and superior accuracy of FiT in distributed low-shot applications including model personalization and federated learning where model update size is an important performance metric.