Computational Perception of Surface Material and 3D Spatial Layout

视觉感知的一项基本任务是估计场景的空间布局，例如，观察者可以移动的地面地形的坡度以及任何障碍物（如物体或墙壁）的位置。解决该布局估计问题的标准计算模型主要集中在具有哑光反射率的场景上，而具有光滑反射表面的场景得到的关注相对较少。重要的例子包括夜间在雨中导航室外街景，这是具有挑战性的，因为这些场景主要是路灯和汽车前灯的镜面高光。其他具有光滑表面的场景示例包括从船上或低空飞行的飞行器上观看的湖泊或海洋表面，或具有抛光地板，光滑油漆或玻璃墙或窗户的室内场景。本研究将讨论如何解释从大的近平面光滑表面产生的镜面高光和镜面反射。该方法将是高度跨学科的，解决机器人视觉、人类视觉感知和计算机图形学中的应用感知问题。应用感知问题的一个关键目标是了解人们在虚拟环境中如何感知光滑的平面布局，这些平面是使用双眼，头部跟踪，宽视场显示器（如Oculus Rift）观看的。这种显示器产生了非凡的沉浸效果，很可能会彻底改变我们体验真实和虚拟世界的方式。本研究应提供洞察这些显示的优势和局限性，以传达空间布局和材料信息的光泽场景。

A fundamental task in visual perception is to estimate the spatial layout of a scene, for example, the slope of the ground terrain on which the observer can move and the positions of any obstacles such an objects or walls. Standard computational models for solving this layout estimation problem have concentrated on scenes that have matte reflectance, but scenes with glossy reflecting surfaces have received relatively little attention. Important examples include navigating outdoor street scenes in the rain at night, which is challenging because such scenes are dominated by specular highlights from street lamps and car head lamps. Other scene examples with glossy surfaces include lake or ocean surfaces that are viewed from a boat or from a low flying aerial vehicle, or indoor scenes with polished floors, glossy painted or glass walls, or windows. This research will address how to interpret the specular highlights and mirror reflections that arise from large near-planar glossy surfaces. The approach will be highly interdisciplinary, addressing problem in robotic vision, human visual perception, and applied perception in computer graphics. A key goal in the applied perception problems in particular is to understand how well people perceive layout of glossy planar surfaces in virtual environments that are viewed using binocular, head-tracked, wide field of view displays such as the Oculus Rift. Such displays yield remarkable immersion effects, and it is likely that they will revolutionize how we experience real and virtual worlds. This research should provide insight into the strengths and limitations of these displays for conveying spatial layout and material information about glossy scenes.