Human Vision: Relationship to Three-Dimensional Surface Statistics of Natural Scenes

经过几代人的进化，人类的视觉系统已经经过了微调，能够在我们特定的环境中有效地运作，使我们能够对周围的物体形成丰富的3D表示。我们每天遇到的场景会产生复杂而模糊的2D视网膜图像。从这些输入中，视觉系统如何以如此令人印象深刻的快速和强大的方式实现恢复我们周围环境的极其困难的目标？为了实现这一壮举，人类必须利用两种关于环境的信息。首先，我们必须学习3D自然场景属性和这些属性产生的2D图像线索之间的概率关系。其次，我们必须了解哪些场景结构（形状、距离、方向）在我们的3D环境中最常见或最可能。这种关于自然3D场景及其投影图像的统计知识使我们能够最大限度地提高我们的感知性能。因此，为了更好地理解3D感知，我们必须研究我们已经进化到可以处理的环境。我们研究的一个关键目标是编目和评估引导人类深度感知的环境的统计结构。我们将对人类经常遇到的场景（不同季节和光照条件下的室内和室外环境）进行采样。对于每个场景，最先进的地面光探测和测距（LiDAR）技术将用于测量从单个位置到所有物体（树木，地面等）的物理距离-场景的3D地图。我们还将从相同的有利位置拍摄相同场景的高动态范围（HDR）照片。通过在众多场景中整理这些配对的3D和2D数据，我们将创建一个我们环境的综合数据库，以及它产生的2D图像。通过公开数据库，它不仅可以促进我们自己的工作，还可以促进世界各地对一系列纯粹和应用视觉过程感兴趣的人类和计算机视觉科学家的研究。计算机视觉有很大的潜力可以从人类视觉系统的专家处理器中学习：计算机视觉算法在一系列任务中很容易被人类超越，特别是当图像对应于更复杂、更真实的场景时。人类视觉系统是如何处理复杂的自然图像的，而这些图像打败了计算机视觉算法。然而，人类视觉系统的鲁棒性似乎取决于：1)利用所有可用的深度线索，2)结合统计“先验”：关于典型场景配置的信息。我们将采用心理物理实验，在我们对自然场景及其图像的分析的指导下，开发有效和全面的人类深度感知的计算模型。我们将集中我们的分析和实验在恢复场景结构的过程中的关键任务-估计的位置，方向和曲面段在整个环境的曲率。我们的项目通过研究大脑如何隐式编码和解释深度信息来指导3D感知，解决了对更复杂和生态有效的人类感知模型的需求。虚拟3D环境现在用于一系列设置，如飞行模拟和训练系统，康复技术，游戏，3D电影和特效。在某些模拟环境中，当视觉输入退化时，感知偏差尤其具有影响力。为了评估和改进这些技术，我们需要更好地理解3D感知。此外，项目所开发的统计模型和推理算法将有助于开发用于自动估计自然场景深度结构的计算机视觉算法。这些算法有许多应用，如2D到3D电影转换，视觉监控和生物识别。

The human visual system has been fine-tuned over generations of evolution to operate effectively in our particular environment, allowing us to form rich 3D representations of the objects around us. The scenes that we encounter on a daily basis produce 2D retinal images that are complex and ambiguous. From this input, how does the visual system achieve the immensely difficult goal of recovering our surroundings, in such an impressively fast and robust way? To achieve this feat, humans must use two types of information about their environment. First, we must learn the probabilistic relationships between 3D natural scene properties and the 2D image cues these produce. Second, we must learn which scene structures (shapes, distances, orientations) are most common, or probable in our 3D environment. This statistical knowledge about natural 3D scenes and their projected images allows us to maximize our perceptual performance. To better understand 3D perception, therefore, we must study the environment that we have evolved to process. A key goal of our research is to catalogue and evaluate the statistical structure of the environment that guides human depth perception. We will sample the range of scenes that humans frequently encounter (indoor and outdoor environments over different seasons and lighting conditions). For each scene, state-of-the-art ground based Light Detection and Ranging (LiDAR) technology will be used to measure the physical distance to all objects (trees, ground, etc.) from a single location - a 3D map of the scene. We will also take High Dynamic Range (HDR) photographs of the same scene, from the same vantage point. By collating this paired 3D and 2D data across numerous scenes we will create a comprehensive database of our environment, and the 2D images that it produces. By making the database publicly available it will facilitate not just our own work, but research by human and computer vision scientists around the world who are interested in a range of pure and applied visual processes.There is great potential for computer vision to learn from the expert processor that is the human visual system: computer vision algorithms are easily out-performed by humans for a range of tasks, particularly when images correspond to more complex, realistic scenes. We are still far from understanding how the human visual system handles the kind of complex natural imagery that defeats computer vision algorithms. However, the robustness of the human visual system appears to hinge on: 1) exploiting the full range of available depth cues and 2) incorporating statistical 'priors': information about typical scene configurations. We will employ psychophysical experiments, guided by our analyses of natural scenes and their images, to develop valid and comprehensive computational models of human depth perception. We will concentrate our analysis and experimentation on key tasks in the process of recovering scene structure - estimating the location, orientation and curvature of surface segments across the environment. Our project addresses the need for more complex and ecologically valid models of human perception by studying how the brain implicitly encodes and interprets depth information to guide 3D perception.Virtual 3D environments are now used in a range of settings, such as flight simulation and training systems, rehabilitation technologies, gaming, 3D movies and special effects. Perceptual biases are particularly influential when visual input is degraded, as they are in some of these simulated environments. To evaluate and improve these technologies we require a better understanding of 3D perception. In addition, the statistical models and inferential algorithms developed in the project will facilitate the development of computer vision algorithms for automatic estimation of depth structure in natural scenes. These algorithms have many applications, such as 2D to 3D film conversion, visual surveillance and biometrics.