A Multigrid Method for Efficiently Training Video Models
A Multigrid Method for Efficiently Training Video Models
复制标题
DOI：
10.1109/cvpr42600.2020.00023
复制
发表时间：
2019-12
期刊：
影响因子：
--
通讯作者：
Chaoxia Wu;Ross B. Girshick;Kaiming He;Christoph Feichtenhofer;Philipp Krahenbuhl
中科院分区：
文献类型：
--
作者：
Chaoxia Wu;Ross B. Girshick;Kaiming He;Christoph Feichtenhofer;Philipp Krahenbuhl
训练有竞争力的深度视频模型比训练其对应的图像模型慢一个数量级。训练缓慢导致研究周期较长，阻碍了视频理解研究的进展。按照训练图像模型的标准做法，视频模型训练使用了固定的小批次形状：特定数量的剪辑、帧和空间大小。然而，最理想的形状是什么？高分辨率模型表现良好，但训练速度较慢。低分辨率模型训练得更快，但精确度较低。受数值优化中多重网格方法的启发，我们提出使用不同时空分辨率的可变小批量形状，这些形状根据时间表而变化。不同的形状是通过对多个采样网格上的训练数据进行重采样而产生的。当缩小其他维度时，通过扩大小批量大小和学习率来加速培训。我们经验地证明了一个通用且健壮的网格时间表，该时间表在不同模型(i3D、非本地、SlowFast)、数据集(Kinetics、Things-Thing、Charade)和训练设置(有或没有预训练、128个GPU或1个GPU)的精度不损失的情况下，产生显著的开箱即用训练加速比。作为一个说明性的例子，所提出的多重网格方法训练ResNet-50 SlowFast网络的速度(挂钟时间，相同的硬件)提高了4.5倍，同时与基准训练相比，Kinetics-400的准确率也提高了0.8%(绝对值)。代码可以在网上获得。
Training competitive deep video models is an order of magnitude slower than training their counterpart image models. Slow training causes long research cycles, which hinders progress in video understanding research. Following standard practice for training image models, video model training has used a fixed mini-batch shape: a specific number of clips, frames, and spatial size. However, what is the optimal shape? High resolution models perform well, but train slowly. Low resolution models train faster, but are less accurate. Inspired by multigrid methods in numerical optimization, we propose to use variable mini-batch shapes with different spatial-temporal resolutions that are varied according to a schedule. The different shapes arise from resampling the training data on multiple sampling grids. Training is accelerated by scaling up the mini-batch size and learning rate when shrinking the other dimensions. We empirically demonstrate a general and robust grid schedule that yields a significant out-of-the-box training speedup without a loss in accuracy for different models (I3D, non-local, SlowFast), datasets (Kinetics, Something-Something, Charades), and training settings (with and without pre-training, 128 GPUs or 1 GPU). As an illustrative example, the proposed multigrid method trains a ResNet-50 SlowFast network 4.5x faster (wall-clock time, same hardware) while also improving accuracy (+0.8% absolute) on Kinetics-400 compared to baseline training. Code is available online.