Visual Prompt Tuning
Visual Prompt Tuning
复制标题
DOI：
10.48550/arxiv.2203.12119
复制
发表时间：
2022-03
期刊：
影响因子：
--
通讯作者：
Menglin Jia;Luming Tang;Bor-Chun Chen;Claire Cardie;Serge J. Belongie;Bharath Hariharan;S. Lim
中科院分区：
文献类型：
--
作者：
Menglin Jia;Luming Tang;Bor-Chun Chen;Claire Cardie;Serge J. Belongie;Bharath Hariharan;S. Lim
目前在调整预训练模型方面的做法涉及更新所有骨干参数，即完全微调。本文介绍了视觉提示调整（VPT）作为一个高效和有效的替代全面微调的大规模Transformer模型的视觉。从最近高效调优大型语言模型的进展中获得灵感，VPT在输入空间中只引入了少量（不到模型参数的1%）可训练参数，同时保持模型主干冻结。通过广泛的实验，各种各样的下游识别任务，我们表明，VPT实现了显着的性能增益相比，其他参数有效的调整协议。最重要的是，在许多情况下，VPT在模型容量和训练数据规模方面甚至优于完全微调，同时降低了每个任务的存储成本。
The current modus operandi in adapting pre-trained models involves updating all the backbone parameters, ie, full fine-tuning. This paper introduces Visual Prompt Tuning (VPT) as an efficient and effective alternative to full fine-tuning for large-scale Transformer models in vision. Taking inspiration from recent advances in efficiently tuning large language models, VPT introduces only a small amount (less than 1% of model parameters) of trainable parameters in the input space while keeping the model backbone frozen. Via extensive experiments on a wide variety of downstream recognition tasks, we show that VPT achieves significant performance gains compared to other parameter efficient tuning protocols. Most importantly, VPT even outperforms full fine-tuning in many cases across model capacities and training data scales, while reducing per-task storage cost.