3d Perception of Specular Surfaces

研究了从图像中恢复反射面形状的方法。同时探讨了计算基础和人类的视觉感知。计算方面的关键问题是：(i)表面形状与表面反射场景观测值之间的几何关系；（ii）有助于从视觉测量中重建形状的约束的性质和作用。第二个问题首先在严格的假设（校准的已知场景）下进行研究，然后逐渐变亮（未校准的已知场景，未校准的未知场景）。附加视觉测量的相关性（来自立体钻机的多幅图像，遮挡边界，内部边界）以及统计约束的相关性和使用（通用视点假设，各向同性，同质性）也进行了探讨。人类对镜子表面的感知很少被探索，也很少被理解。第一个研究领域是在越来越多的线索（图像补丁、表面边界、反射场景、内部边界）和不同场景统计（自然和合成的规则周期图案、各向同性纹理、室内场景、室外场景）的存在下进行定性形状感知。接下来将探讨导致人类视觉系统将表面分类为镜面与纹理/哑光的线索。第三个问题是纹理形状和镜面形状的机制之间的关系。提出的研究提供了：i)测量镜面形状的方法，这是计算机视觉中众所周知的难题；Ii)对反射表面视觉的几何和统计学的基本理解；iii)在贝叶斯框架中探索先验知识的价值；Iv)对人类视觉系统未被充分开发的能力的洞察。该提案的更广泛影响包括将3D扫描系统的适用性扩展到高光表面，这在工程，医学和艺术保护中很常见。因此，通用、实用和低成本的方法是本研究特别感兴趣的。

The recovery of the shape of reflective surfaces from images is investigated. Both the computational foundations as well as human visual perception are explored. Key issues on the computational front are : (i) the geometrical relationship between surface shape and observations of a reflected scene on the surface; (ii) the nature and role of constraints that help in reconstructing shape from visual measurements. This second issue is investigated first under stringent assumptions (calibrated known scene) which are then lightened progressively (un-calibrated known scene, un-calibrated unknown scene). The relevance of additional visual measurements (multiple images from stereoscopic rigs, occluding boundaries, internal boundaries) as well as the relevance and use of statistical constraints (generic viewpoint assumption, isotropy, homogeneity) is also explored. Human perception of mirror surfaces is little explored and very poorly understood. The first area of investigation is qualitative shape perception in the presence of an increasing number of cues (image patch, surface boundaries, reflected scene, internal boundaries) and with different scene statistics both natural and synthetic (regular periodic patterns, isotropic textures, indoor scenes, outdoor scenes). The cues that lead the human visual system to classify surfaces as specular vs. textured/matte are explored next. A third issue is the relationship between the mechanisms underlying shape-from-texture and shape-from-specularities. The proposed research provides: i) methods to measure the shape of specular surfaces, a notoriously hard problem in computer vision; ii) fundamental understanding of the geometry and statistics underlying vision of reflective surfaces; iii) exploration of the value of prior knowledge in a Bayesian framework; iv) insight into an underexplored ability of the human visual system. The broader impacts of this proposal include extending the applicability of 3D scanning system to specular surfaces, which are common in engineering, medicine and art conservation. For this reason methods that are general, practical and low-cost are of particular interest in this study.