Semantic Structure from Multiple Uncalibrated Images

图像包含了大量的信息，这些信息通常可以立即被人类破译。 尽管到目前为止取得了显着的进展，人工视觉系统的图像理解仍然有限。因此，让机器像人类一样看东西仍然是计算机视觉研究的最终目标。 本研究旨在推进当前的视觉系统，使其能够从一个或多个未校准的图像中推断出2D和/或3D高级语义信息。 具体而言，将研究以下问题。 (1)平行度、不平行度等几何信息的自动检测。这些几何关系可以在图像理解中发挥核心作用，因为它们允许推断其他更高级别的信息。我们最近在并行的自动检测上取得了一些成功 平面从多个未校准的图像，我们计划以其他几何信息为目标。 (2)语义几何信息和摄像机自动标定之间的关系，用于使用多幅图像推断观察场景的完整3D度量信息。我们的目标是开发一个框架，不同的高层次的几何关系，可以使用在一个灵活的方式作为约束的相机参数和/或整个3D结构的观察到的场景。当图像由不同的未知相机拍摄时，这一点尤其重要。 (3)将场景语义分割为目标，这一问题的解决将使机器视觉系统更接近人类。来自不同来源的不同线索可以用来帮助这种分割。除了颜色分割之外，3D度量信息和高级几何关系可以一起用于成功的语义分割。

Images contain considerable amount of information that is usually immediately deciphered by humans. Despite the remarkable progress made so far, image understanding by artificial vision systems remains limited. Hence, making machines see as humans do is still an ultimate goal for computer vision research. This research proposal aims at advancing current vision systems to make them capable of inferring 2D and/or 3D high level semantic information from one or multiple uncalibrated images. In particular, the following problems will be investigated. (1) The automatic detection of geometric information such as parallelism and perpendicularity. These geometric relationships can play a central role in image understanding as they allow to infer other higher level information. We have recently achieved some success on the automatic detection of parallel planes from multiple non-calibrated images and we plan to target other geometric information. (2) The relationship between semantic geometric information and camera auto-calibration for inferring full 3D metric information of the observed scene using multiple images. We aim at developing a framework where different high level geometric relationships can be used in a flexible way as constraints on the camera parameters and/or the whole 3D structure of the observed scene. This is especially important when images have been taken by different unknown cameras. (3) The semantic segmentation of the scene into objects, a problem if solved will make the machine vision system one step closer to the human one. Different clues coming from different sources can be used to help this segmentation. In addition to the color segmentation, the 3D metric information and the high level geometric relationships could used together for a successful semantic segmentation.