Computational Perception of Surface Material and 3D Spatial Layout

视觉感知的一个基本任务是估计场景的空间布局，例如，观察者可以移动的地面地形的坡度以及任何障碍物（如物体或墙壁）的位置。用于解决该布局估计问题的标准计算模型集中在具有哑光反射率的场景上，但是具有光泽反射表面的场景受到的关注相对较少。重要的示例包括在夜间雨中导航户外街道场景，这是具有挑战性的，因为这些场景由来自路灯和汽车前灯的镜面高光主导。具有光泽表面的其他场景示例包括从船上或从低空飞行的飞行器上查看的湖泊或海洋表面，或具有抛光地板、光泽涂漆或玻璃墙或窗户的室内场景。这项研究将解决如何解释镜面高光和镜面反射，从大的近平面光泽表面产生。该方法将是高度跨学科的，解决机器人视觉，人类视觉感知和计算机图形学中的应用感知问题。应用感知问题中的一个关键目标特别是要了解人们如何感知虚拟环境中光滑平面表面的布局，这些虚拟环境使用双目，头部跟踪，宽视场显示器（如Oculus Rift）进行查看。这种显示器产生了显著的沉浸效果，很可能会彻底改变我们体验真实的和虚拟世界的方式。这项研究应该提供洞察这些显示器的优势和局限性，传达空间布局和材料信息光泽的场景。

A fundamental task in visual perception is to estimate the spatial layout of a scene, for example, the slope of the ground terrain on which the observer can move and the positions of any obstacles such an objects or walls. Standard computational models for solving this layout estimation problem have concentrated on scenes that have matte reflectance, but scenes with glossy reflecting surfaces have received relatively little attention. Important examples include navigating outdoor street scenes in the rain at night, which is challenging because such scenes are dominated by specular highlights from street lamps and car head lamps. Other scene examples with glossy surfaces include lake or ocean surfaces that are viewed from a boat or from a low flying aerial vehicle, or indoor scenes with polished floors, glossy painted or glass walls, or windows. This research will address how to interpret the specular highlights and mirror reflections that arise from large near-planar glossy surfaces. The approach will be highly interdisciplinary, addressing problem in robotic vision, human visual perception, and applied perception in computer graphics. A key goal in the applied perception problems in particular is to understand how well people perceive layout of glossy planar surfaces in virtual environments that are viewed using binocular, head-tracked, wide field of view displays such as the Oculus Rift. Such displays yield remarkable immersion effects, and it is likely that they will revolutionize how we experience real and virtual worlds. This research should provide insight into the strengths and limitations of these displays for conveying spatial layout and material information about glossy scenes.