Efficient Active Online Learning for 3D Reconstruction and Scene Understanding

机器学习算法已经成为现代计算机视觉系统的重要组成部分。然而，这些系统中的大多数都是基于离线学习的，即在系统部署之前只进行一次学习，并且没有考虑将来对未观察到的情况的适应。此外，学习任务所需的标记训练数据量通常非常高，因为系统不可能选择特别适合所需分类任务的训练数据子集。相反，人类的感知强烈依赖于我们不断学习的能力，并使我们获得的知识适应新的环境和环境。该项目的目标是通过开发新的学习方法来解决这一挑战，这种学习方法比当前最先进的系统具有更高程度的自主性。在这里，自主性是指系统决定哪种信息对更有效的学习更有用的能力。为了实现这一点，我们将开发一个主动学习系统，在这个系统中，学习是循环完成的：在初始训练阶段之后，系统被提供新的观察值用于分类。它从中选择那些信息量最大的信息。为此，从人那里查询标签，并将获得的标记数据用于下一轮训练。这将导致对当前系统的两个主要改进：首先，所需的标记训练数据量将显着降低，因为使用的训练数据将更加面向上下文。其次，系统将具有更高的适应新情况或环境的能力，因为学习是在一个持续的过程中完成的。我们的主动学习系统将应用于计算机视觉中的两个重要研究挑战，即3D重建和场景理解，目的是使用我们提出的自主，即在这种情况下的主动学习方法，在这些领域实现实质性的性能改进。

Machine learning algorithms have become an important building block in modern computer vision systems. However, most of these systems are based on offline learning, i.e. learning is done only once before system deployment, and future adaptations to unobserved circumstances are not considered. Furthermore, the amount of labeled training data required for the learning task is usually very high, because the system has no possibility to select subsets of the training data that are particularly suited for the required classification task. Quite in contrast, human perception strongly relies on our capacity to constantly learn and adapt our acquired knowledge to new environments and circumstances. The goal of this project is to address this challenge by developing novel learning methods that perform the learning task with a higher degree of autonomy than current state-of-the-art systems. Here, autonomy refers to the ability of the system to decide which kind of information is more useful for more efficient learning. To achieve this, we will develop an Active Learning system, in which learning is done in cycles: After an initial training phase the system is presented with new observations for classification. Among them it selects those that are most infomative. For these, labels are queried from a human and the so obtained labeled data are used in the next round of training. This will lead to two major improvements over current systems: first, the required amount of labeled training data will be significantly lower, because the used training data will be much more context-oriented. And second, the system will have a much higher capability to adapt to new situations or environments, because learning is done in an ongoing process. Our Active Learning system will be applied to two important research challenges in computer vision, namely 3D reconstruction and scene understanding, and the aim is to achieve substantial performance improvements in these areas using our proposed autonomous, i.e. in this case active learningapproach.