End-to-End optimized image compression for machines, a study
End-to-End optimized image compression for machines, a study
复制标题
机器的端到端优化图像压缩，一项研究
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Simon Feltman
中科院分区：
文献类型：
--
作者：
Lahiru D. Chamain;Fabien Racapé;Jean Bégaint;Akshay Pushparaja;Simon Feltman
越来越多的图像和视频内容是由机器分析的，而不是由人类观看的，因此，对于远程执行分析的此类应用，优化编解码器变得至关重要。不幸的是，传统的编码工具具有挑战性，专门用于机器任务，因为它们最初是为人类感知而设计的。然而，基于神经网络的编解码器可以与任何基于卷积神经网络（CNN）的任务模型端到端联合训练。在本文中，我们建议研究一个端到端的框架，使有效的图像压缩远程机器任务分析，使用一个链组成的压缩模块和任务算法，可以优化端到端。我们表明，它是可能的，以显着提高任务的准确性时，微调联合编解码器和任务网络，特别是在低比特率。根据训练或部署约束，选择性微调可以仅应用于编码器、解码器或任务网络，并且仍然可以实现比现成的编解码器和任务网络更高的速率精度。我们的研究结果还证明了端到端管道在实际应用中的灵活性。
An increasing share of image and video content is analyzed by machines rather than viewed by humans, and therefore it becomes relevant to optimize codecs for such applications where the analysis is performed remotely. Unfortunately, conventional coding tools are challenging to specialize for machine tasks as they were originally designed for human perception. However, neural network based codecs can be jointly trained end-to-end with any convolutional neural network (CNN)-based task model. In this paper, we propose to study an end-to-end framework enabling efficient image compression for remote machine task analysis, using a chain composed of a compression module and a task algorithm that can be optimized end-to-end. We show that it is possible to significantly improve the task accuracy when fine-tuning jointly the codec and the task networks, especially at low bitrates. Depending on training or deployment constraints, selective fine-tuning can be applied only on the encoder, decoder or task network and still achieve rate-accuracy improvements over an off-the-shelf codec and task network. Our results also demonstrate the flexibility of end-to-end pipelines for practical applications.