Accel: A Corrective Fusion Network for Efficient Semantic Segmentation on Video
Accel: A Corrective Fusion Network for Efficient Semantic Segmentation on Video
复制标题
DOI：
10.1109/cvpr.2019.00907
复制
发表时间：
2018-07
期刊：
影响因子：
--
通讯作者：
Samvit Jain;Xin Wang;Joseph E. Gonzalez
中科院分区：
文献类型：
--
作者：
Samvit Jain;Xin Wang;Joseph E. Gonzalez
我们提出了Accel，这是一种新型的语义视频分割系统，通过结合两个网络分支的预测，以低推理成本实现了高精度：（1）参考分支，其提取参考关键帧上的高细节特征，并且使用帧到帧光流估计来向前扭曲这些特征，以及（2）更新分支，其计算当前帧上的可调节质量的特征，在每个视频帧处执行时间更新。更新分支的模块化（其中可以插入不同层深度的特征子网络（例如ResNet-18到ResNet-101））使得能够在新的、最先进的精度-吞吐量折衷频谱上进行操作。在这条曲线上，Accel模型比最接近的可比单帧分割网络实现了更高的准确性和更快的推理时间。总的来说，Accel在有效的语义视频分割方面明显优于以前的工作，纠正了在具有复杂动态的数据集上复合的扭曲相关错误。Accel是端到端可训练的，高度模块化：参考网络、光流网络和更新网络都可以根据应用需求独立选择，然后联合进行微调。其结果是一个强大的，通用的系统，快速，高精度的视频语义分割。
We present Accel, a novel semantic video segmentation system that achieves high accuracy at low inference cost by combining the predictions of two network branches: (1) a reference branch that extracts high-detail features on a reference keyframe, and warps these features forward using frame-to-frame optical flow estimates, and (2) an update branch that computes features of adjustable quality on the current frame, performing a temporal update at each video frame. The modularity of the update branch, where feature subnetworks of varying layer depth can be inserted (e.g. ResNet-18 to ResNet-101), enables operation over a new, state-of-the-art accuracy-throughput trade-off spectrum. Over this curve, Accel models achieve both higher accuracy and faster inference times than the closest comparable single-frame segmentation networks. In general, Accel significantly outperforms previous work on efficient semantic video segmentation, correcting warping-related error that compounds on datasets with complex dynamics. Accel is end-to-end trainable and highly modular: the reference network, the optical flow network, and the update network can each be selected independently, depending on application requirements, and then jointly fine-tuned. The result is a robust, general system for fast, high-accuracy semantic segmentation on video.