Video Coding for Deep Learning-Based Machine-to-Machine Communication

在本研究项目中，视频编码的机器（VCM）的任务将研究与最近出现的神经压缩网络（NCN）的强烈关注。在那里，自动编码器网络被训练成从需要尽可能少的比特率来传输的潜在空间重建输入图像。在后续项目中，将以最先进的针对人类优化的NCN作为起点，为VCM任务进行自己的优化和设计。为此，该项目分为两个阶段。首先，帧内压缩的研究，通过开发新的训练损失表示在解码器侧的评估网络，以提高VCM的编码增益。因此，将在通常针对多个评估网络进行优化与特定评估网络在编码之前已知且可用时进行区分。第二，从第一阶段发现的方法应该被用来进一步优化为帧间编码设计的合适的NCN架构。然而，合适的原始标记的视频数据必须被采集用于之前的适当评估。此外，还应考虑需要视频数据（如跟踪）的测试用例，以衡量编码效率。

In this research project, the video coding for machines (VCM) task shall be researched with a strong focus on the recently emerged neural compression networks (NCNs). There, autoencoder networks are trained to reconstruct the input image from a latent space requiring as as little bitrate as possible to transmit. For the follow-up project, state-of-the-art NCNs optimized for the human shall serve as a starting point to base own optimizations and designs for the VCM task on. To that end, the project is divided into two phases. First, intra compression is researched by developing novel training losses representing the evaluation network at the decoder side in order to improve the coding gains for VCM. Thereby, it will be differentiated between generally optimizing for multiple evaluation networks and when the specific evaluation network is known and available before encoding. Second, the found methods from the first phase are supposed to be used to further optimize suitable NCN architectures designed for inter coding. However, suitable pristine labeled video data has to be acquired for an appropriate evaluation before. Besides, also test cases requiring video data such as tracking shall be considered to measure the coding efficiency.