Training Deep Networks for Real-world Computer Vision Scenarios with Rendered Data

用深度学习来处理计算机视觉任务需要大数据集来充分发挥这种方法的潜力。然而，对于一些计算机视觉任务，如运动估计或深度估计，不可能通过人工标注图像来提供如此大的数据集。一种可能的解决方案是使用合成的渲染图像数据。在这种情况下，输入图像和所需输出都被渲染，并且可用于训练过程。然而，归根结底，网络应该会在真实数据上提供良好的结果。在这个项目中，我们想要开发的方法将使我们能够在合成数据上训练网络，同时也能在真实数据上取得最佳结果。为此，我们将开发一些方法，将除带注释的合成数据外的未带注释的真实数据集成到训练过程中。重点将放在视频的光流估计和深度估计上。此外，我们还想调查开发的概念在多大程度上可以用于基于图像的控制。

Approaching computer vision tasks with deep learning requires large datasets to exploit the full potential of this approach. For some computer vision tasks, such as motion estimation or depth estimation, however, it is impossible to provide such large datasets by manual annotation of images. One possible solution is the use of synthetic, rendered image data. In this case, both the input images and the desired outputs are rendered and are available for the training process. However, in the end, the network should provide good results on real data. In this project we want to develop methods that will allow us to train networks on synthetic data but also achieve optimal results on real data. To this end, we will develop methods that integrate non-annotated real data besides the annotated synthetic data into the training process. The focus will be on optical flow estimation and depth estimation from videos. Moreover, we want to investigate to what extent the developed concepts can be employed for image based control, too.