Visual Computation of Scene Layout

3D视觉感知的一个基本挑战是，我们从自己以自我为中心的视角观察世界，但我们希望了解世界的实际情况，不受自己视角的影响。我的长期研究目标是展示人类和计算机视觉系统如何能够从各种视觉线索的信息中感知世界的3D几何形状。我特别尝试识别新的视觉线索，并在此基础上开发计算模型。我的短期目标是解决类似景观的场景。许多人类视觉研究表明，户外场景中的3D空间感知存在系统性偏差：较大的距离看起来是空间压缩的，斜坡表面（如山丘和山谷）看起来比实际更陡峭。基于深度学习的估计深度的计算机视觉系统也有偏差，这取决于它们所训练的数据。在本研究中，我们将研究人类观察者和计算机视觉系统对空间布局的估计；具体来说，我们将研究一个被忽视的场景，即“地平面”是非平面的场景。对于人类视觉研究，我们将使用虚拟现实显示器来测量人类对深度和地平面斜率的感知。我们将概括许多以前的研究中使用的方法，这些研究几乎都检查了平面地平面的情况。对于计算机视觉的研究，我们将使用现有的基于深度神经网络的深度估计方法，并测量估计深度的偏差和不确定性。通过比较人类和计算机的视觉表现，我们希望更好地了解这两种视觉系统使用的信息。学员将获得一系列技能，包括如何进行用户研究，如何使用VR显示器，以及如何使用深度学习计算机视觉方法来估计场景布局。这将为他们提供进入新兴领域所需的专业知识，例如VR内容创作和自动驾驶汽车的视觉系统。

A fundamental challenge of 3D visual perception is that we observe the world from our own self-centered viewpoint, and yet we wish to know how the world actually is, unbiased by our viewpoint. My long term research objective is to show how human and computer vision systems are able to perceive the 3D geometry of the world, from information in various visual cues. I specifically try to identify novel visual cues and develop computational models based on them. My short term objective is to address the case of landscape-like scenes. Many human vision studies have shown that 3D space perception in outdoor scenes suffers from systematic biases: large distances appear spatially compressed, and sloped surfaces such as hills and valleys appear steeper than they are. Deep learning based computer vision systems for estimating depth also have biases, depending on the data they are trained on. In this proposed research, we will examine how well human observers and computer vision systems can estimate spatial layout; specifically we will examine a neglected scenario of landscape-like scenes in which the `ground plane' is non-planar. For the human vision studies, we will use virtual reality displays to measure human perception of depth and ground plane slope. We will generalize methodologies used in many previous studies which have nearly all examined the case of planar ground planes. For the computer vision studies, we will use existing deep neural network-based depth estimation methods, and measure bias and uncertainty in estimated depth. By comparing human and computer vision performance, we hope to better understand what information both types of vision systems are using. Trainees will acquire a range of skills, including how to do user studies, how to use VR displays, and how to use deep learning computer vision methods to estimate scene layout. This will give them the expertise needed to enter emerging fields, such as VR content creation, and vision systems for self-driving cars.