Semantic Structure from Multiple Uncalibrated Images

图像包含相当多的信息，这些信息通常可以立即被人类破译。 尽管到目前为止已经取得了显著的进步，但人工视觉系统对图像的理解仍然有限。因此，让机器像人类一样看东西仍然是计算机视觉研究的终极目标。 该研究方案旨在改进现有的视觉系统，使其能够从一个或多个未校准的图像中推断2D和/或3D高级语义信息。特别是，将调查以下问题。 (1)平行度、垂直度等几何信息的自动检测。这些几何关系可以在图像理解中发挥核心作用，因为它们允许推断其他更高级别的信息。我们最近在并行的自动检测方面取得了一些成功 来自多个未校准图像的飞机，我们计划以其他几何信息为目标。 (2)语义几何信息与摄像机自动标定之间的关系，利用多幅图像推断观测场景的全部三维度量信息。我们的目标是开发一个框架，其中可以灵活地使用不同的高级几何关系作为对摄像机参数和/或观察场景的整个3D结构的约束。当图像是由不同的未知相机拍摄的时候，这一点尤其重要。 (3)场景到对象的语义分割，这一问题的解决将使机器视觉系统更接近人类视觉系统。来自不同来源的不同线索可以帮助这种分割。除了颜色分割外，3D度量信息和高层几何关系可以共同用于成功的语义分割。

Images contain considerable amount of information that is usually immediately deciphered by humans. Despite the remarkable progress made so far, image understanding by artificial vision systems remains limited. Hence, making machines see as humans do is still an ultimate goal for computer vision research. This research proposal aims at advancing current vision systems to make them capable of inferring 2D and/or 3D high level semantic information from one or multiple uncalibrated images. In particular, the following problems will be investigated. (1) The automatic detection of geometric information such as parallelism and perpendicularity. These geometric relationships can play a central role in image understanding as they allow to infer other higher level information. We have recently achieved some success on the automatic detection of parallel planes from multiple non-calibrated images and we plan to target other geometric information. (2) The relationship between semantic geometric information and camera auto-calibration for inferring full 3D metric information of the observed scene using multiple images. We aim at developing a framework where different high level geometric relationships can be used in a flexible way as constraints on the camera parameters and/or the whole 3D structure of the observed scene. This is especially important when images have been taken by different unknown cameras. (3) The semantic segmentation of the scene into objects, a problem if solved will make the machine vision system one step closer to the human one. Different clues coming from different sources can be used to help this segmentation. In addition to the color segmentation, the 3D metric information and the high level geometric relationships could used together for a successful semantic segmentation.