Semantic Structure from Multiple Uncalibrated Images

点击翻译按钮获取中文摘要

Images contain considerable amount of information that is usually immediately deciphered by humans. Despite the remarkable progress made so far, image understanding by artificial vision systems remains limited. Hence, making machines see as humans do is still an ultimate goal for computer vision research. This research proposal aims at advancing current vision systems to make them capable of inferring 2D and/or 3D high level semantic information from one or multiple uncalibrated images. In particular, the following problems will be investigated. (1) The automatic detection of geometric information such as parallelism and perpendicularity. These geometric relationships can play a central role in image understanding as they allow to infer other higher level information. We have recently achieved some success on the automatic detection of parallel planes from multiple non-calibrated images and we plan to target other geometric information. (2) The relationship between semantic geometric information and camera auto-calibration for inferring full 3D metric information of the observed scene using multiple images. We aim at developing a framework where different high level geometric relationships can be used in a flexible way as constraints on the camera parameters and/or the whole 3D structure of the observed scene. This is especially important when images have been taken by different unknown cameras. (3) The semantic segmentation of the scene into objects, a problem if solved will make the machine vision system one step closer to the human one. Different clues coming from different sources can be used to help this segmentation. In addition to the color segmentation, the 3D metric information and the high level geometric relationships could used together for a successful semantic segmentation.