Scene manipulation in the context of uncalibrated images

点击翻译按钮获取中文摘要

Images contain a considerable amount of rich and concentrated information that is usually deciphered immediately by humans. In comparison and despite the remarkable progress made so far, image understanding by artificial vision systems remains limited. This research proposal aims at advancing current vision systems to make them capable of manipulating parts of a scene, where the latter is represented by a sequence of two-dimensional uncalibrated images. In particular, we will deal with the following three problems: adding new objects(augmented reality), removing existing objects and, moving/changing objects within the scene. In this proposal, we consider the difficult case where images are taken with unknown cameras and the only data to be used are the arrays of pixels. Furthermore, the solutions to be proposed here will be all vision-based, with no need for specialized equipment. In addition to the scientific advancement aspect, the results from this research could be applied in areas ranging from teleoperation to entertainment. More precisely, we will investigate the following related problems.(1) Registration of virtual objects in the images. Currently, some vision-based solutions, which use calibrated cameras exist for this problem. However, the registration becomes complex when the only data we have consist of uncalibrated images. In this case, we need to retrieve some metric (Euclidean) information about the observed scene in order to achieve correct registration. We will investigate the two possibilities to get such metric information: either by using recent techniques for camera self-calibration or by using Euclidean constraints from the scene/object geometry.(2) Segmentation and object tracking across images. Removing or displacing objects requires proper segmentation and identification of the target objects. The latter should be identified and matched across the images, in order to be able to move it to a different location.(3) Updating the whole image sequence. Once the first two images have been changed, either by the addition of new objects or by the removal/displacement of existing objects, the rest of the sequence must be updated accordingly.