3d Perception of Specular Surfaces

研究了从图像中恢复反射面形状的问题。这两个计算基础以及人类的视觉感知进行了探索。计算前沿的关键问题是：（i）表面形状和表面上反射场景的观测之间的几何关系;（ii）有助于从视觉测量重建形状的约束的性质和作用。这第二个问题首先在严格的假设下（校准的已知场景），然后逐步减轻（未校准的已知场景，未校准的未知场景）。额外的视觉测量的相关性（多个图像从立体钻机，闭塞的边界，内部边界），以及相关性和使用的统计约束（通用的观点假设，各向同性，同质性）也进行了探讨。人类对镜面的感知很少被探索，也很少被理解。调查的第一个领域是定性的形状感知的存在下，越来越多的线索（图像补丁，表面边界，反射场景，内部边界）和不同的场景统计自然和合成（定期的周期性图案，各向同性纹理，室内场景，室外场景）。接下来将探讨导致人类视觉系统将表面分类为镜面反射与纹理/哑光的线索。第三个问题是形状从纹理和形状从镜面反射的机制之间的关系。拟议的研究提供：i）测量镜面形状的方法，这是计算机视觉中的一个众所周知的难题; ii）对反射表面视觉的几何和统计基础的基本理解; iii）在贝叶斯框架中探索先验知识的价值; iv）洞察人类视觉系统的未充分开发的能力。该提案的更广泛影响包括将3D扫描系统的适用性扩展到镜面，这在工程，医学和艺术保护中很常见。因此，本研究特别关注通用、实用和低成本的方法。

The recovery of the shape of reflective surfaces from images is investigated. Both the computational foundations as well as human visual perception are explored. Key issues on the computational front are : (i) the geometrical relationship between surface shape and observations of a reflected scene on the surface; (ii) the nature and role of constraints that help in reconstructing shape from visual measurements. This second issue is investigated first under stringent assumptions (calibrated known scene) which are then lightened progressively (un-calibrated known scene, un-calibrated unknown scene). The relevance of additional visual measurements (multiple images from stereoscopic rigs, occluding boundaries, internal boundaries) as well as the relevance and use of statistical constraints (generic viewpoint assumption, isotropy, homogeneity) is also explored. Human perception of mirror surfaces is little explored and very poorly understood. The first area of investigation is qualitative shape perception in the presence of an increasing number of cues (image patch, surface boundaries, reflected scene, internal boundaries) and with different scene statistics both natural and synthetic (regular periodic patterns, isotropic textures, indoor scenes, outdoor scenes). The cues that lead the human visual system to classify surfaces as specular vs. textured/matte are explored next. A third issue is the relationship between the mechanisms underlying shape-from-texture and shape-from-specularities. The proposed research provides: i) methods to measure the shape of specular surfaces, a notoriously hard problem in computer vision; ii) fundamental understanding of the geometry and statistics underlying vision of reflective surfaces; iii) exploration of the value of prior knowledge in a Bayesian framework; iv) insight into an underexplored ability of the human visual system. The broader impacts of this proposal include extending the applicability of 3D scanning system to specular surfaces, which are common in engineering, medicine and art conservation. For this reason methods that are general, practical and low-cost are of particular interest in this study.