Visual Computation of Scene Layout

场景布局可视化计算

• 批准号: RGPIN-2022-03489
• 负责人: Langer, Michael
• 金额: $ 1.75万
• 依托单位: McGill University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=750366
• 关键词: Visual; Computation; Scene; Layout

A fundamental challenge of 3D visual perception is that we observe the world from our own self-centered viewpoint, and yet we wish to know how the world actually is, unbiased by our viewpoint. My long term research objective is to show how human and computer vision systems are able to perceive the 3D geometry of the world, from information in various visual cues. I specifically try to identify novel visual cues and develop computational models based on them. My short term objective is to address the case of landscape-like scenes. Many human vision studies have shown that 3D space perception in outdoor scenes suffers from systematic biases: large distances appear spatially compressed, and sloped surfaces such as hills and valleys appear steeper than they are. Deep learning based computer vision systems for estimating depth also have biases, depending on the data they are trained on. In this proposed research, we will examine how well human observers and computer vision systems can estimate spatial layout; specifically we will examine a neglected scenario of landscape-like scenes in which the `ground plane' is non-planar. For the human vision studies, we will use virtual reality displays to measure human perception of depth and ground plane slope. We will generalize methodologies used in many previous studies which have nearly all examined the case of planar ground planes. For the computer vision studies, we will use existing deep neural network-based depth estimation methods, and measure bias and uncertainty in estimated depth. By comparing human and computer vision performance, we hope to better understand what information both types of vision systems are using. Trainees will acquire a range of skills, including how to do user studies, how to use VR displays, and how to use deep learning computer vision methods to estimate scene layout. This will give them the expertise needed to enter emerging fields, such as VR content creation, and vision systems for self-driving cars.

3D视觉感知的一个基本挑战是，我们从自我中心的角度观察世界，但我们希望知道世界实际上是如何的，不受我们观点的影响。我的长期研究目标是展示人类和计算机视觉系统如何能够从各种视觉线索中感知世界的3D几何形状。我特别尝试识别新颖的视觉线索，并在此基础上开发计算模型。我的短期目标是解决类似牺牲的场景的情况。许多人类视觉研究表明，户外场景中的3D空间感知受到系统性偏差的影响：大距离看起来在空间上被压缩，倾斜的表面（如山丘和山谷）看起来比实际更陡峭。基于深度学习的计算机视觉系统估计深度也有偏差，这取决于它们训练的数据。在这项拟议的研究中，我们将研究人类观察者和计算机视觉系统如何估计空间布局;具体来说，我们将研究一个被忽视的场景，即“地平面”是非平面的场景。对于人类视觉研究，我们将使用虚拟现实显示器来测量人类对深度和地平面坡度的感知。我们将概括的方法，在许多以前的研究，几乎所有的情况下，平面接地层。对于计算机视觉研究，我们将使用现有的基于深度神经网络的深度估计方法，并测量估计深度的偏差和不确定性。通过比较人类和计算机的视觉性能，我们希望更好地了解这两种类型的视觉系统都使用了哪些信息。学员将获得一系列技能，包括如何进行用户研究，如何使用VR显示器，以及如何使用深度学习计算机视觉方法来估计场景布局。这将为他们提供进入新兴领域所需的专业知识，例如VR内容创作和自动驾驶汽车视觉系统。