Computational Perception of Surface Material and 3D Spatial Layout

视觉感知的一项基本任务是估计场景的空间布局，例如，观察者可以在其上移动的地面地形的坡度，以及任何障碍物(如物体或墙壁)的位置。解决布局估计问题的标准计算模型主要集中在具有无光反射的场景，而具有光泽反射表面的场景受到的关注相对较少。重要的例子包括在夜间雨中导航户外街道场景，这是具有挑战性的，因为此类场景主要是路灯和汽车前大灯的镜面高光。其他具有光泽表面的场景示例包括从船上或从低空飞行的飞行器上观看的湖泊或海洋表面，或者具有抛光地板、光滑的油漆或玻璃墙或窗户的室内场景。这项研究将解决如何解释由大的近平面光泽表面产生的镜面高光和镜面反射。该方法将是高度跨学科的，解决机器人视觉、人类视觉感知和计算机图形学中的应用感知的问题。应用感知问题的一个关键目标尤其是了解人们对虚拟环境中使用双目、头部跟踪、宽视场显示器(如Oculus Rift)查看的光滑平面的布局的感知程度。这样的展示产生了显著的沉浸效果，很可能会彻底改变我们体验现实和虚拟世界的方式。这项研究应该能洞察到这些显示器在传达空间布局和关于光泽场景的材料信息方面的优势和局限性。

A fundamental task in visual perception is to estimate the spatial layout of a scene, for example, the slope of the ground terrain on which the observer can move and the positions of any obstacles such an objects or walls. Standard computational models for solving this layout estimation problem have concentrated on scenes that have matte reflectance, but scenes with glossy reflecting surfaces have received relatively little attention. Important examples include navigating outdoor street scenes in the rain at night, which is challenging because such scenes are dominated by specular highlights from street lamps and car head lamps. Other scene examples with glossy surfaces include lake or ocean surfaces that are viewed from a boat or from a low flying aerial vehicle, or indoor scenes with polished floors, glossy painted or glass walls, or windows. This research will address how to interpret the specular highlights and mirror reflections that arise from large near-planar glossy surfaces. The approach will be highly interdisciplinary, addressing problem in robotic vision, human visual perception, and applied perception in computer graphics. A key goal in the applied perception problems in particular is to understand how well people perceive layout of glossy planar surfaces in virtual environments that are viewed using binocular, head-tracked, wide field of view displays such as the Oculus Rift. Such displays yield remarkable immersion effects, and it is likely that they will revolutionize how we experience real and virtual worlds. This research should provide insight into the strengths and limitations of these displays for conveying spatial layout and material information about glossy scenes.