A test of a novel non-probabilistic model of 3D cue integration

尽管进行了至少一个世纪的科学研究，但人类大脑如何从通过光线到达眼睛的2D信息中构建对三维(3D)物体和空间的感知仍然不完全清楚。  很早以前就知道，这种能力是在各种不同的视觉信号(大脑解释)的基础上获得的，这种信号被大脑组合在一起，得出场景的3D结构。  深度线索的例子可以在绘画中观察到，其中线性透视，阴影，即使是简单的轮廓也在平面画布上描绘了3D世界，就像人们在观察真实3D对象时在平面人类视网膜上所做的那样。然而，这些线索如何编码3D信息，以及不同的线索是如何组合在一起，以产生我们对3D视觉空间和物体的稳定一致的感知，仍然知之甚少。  科学文献中的主流理论(贝叶斯概率推理理论)假设，大脑通过确定特定3D结构在视网膜上获得信息的可能性来推导出世界的3D结构。实现这一模型需要一系列假设，比如深度线索提供了对3D参数的“嘈杂”估计，平均而言，这些估计仍然是准确的。然而，许多常见和重要的观测结果不能用贝叶斯模型完全解释，并使人对该模型的关键计算假设产生怀疑。此外，贝叶斯模型努力解释我们在观看真实世界和人工场景(如图画图像、虚拟或增强现实(VR、AR))之间在三维“质量”方面的重要差异。该项目测试了一种新理论(本征约束理论)，与贝叶斯理论相比，该理论做出了一组完全不同且更简单的假设，但它可以预测更广泛的感知现象。  该项目将IC理论与最近提出的理论相结合，该理论假设视觉系统不产生3D空间的单一编码，而是产生两种不同的编码，其中一种与理解场景(即，3D对象形状和布局)和另一种基础视觉引导的运动，如伸手和抓住对象。  后一种编码声称是在观看立体图像(例如，3D电影)时最明显的3维的特殊主观体验的基础。该项目将表明，IC模型可以有效地整合3D空间的两种不同表示形式的主张。通过这样做，它能够更好地解释我们对三维感知的一系列基本方面，这些方面很难用主流模型来解释，包括更好地理解对开发3D技术(例如，VR和AR)重要的因素所需的那些方面。该奖项支持一项实证研究，该研究测试了由线索整合的内在约束理论产生的计算模型，以及属于贝叶斯框架的主流计算模型。具体地说，新模型挑战了贝叶斯模型的三个主要假设，这些假设将在不同的工作包中进行实验测试： (1)深度线索平均提供真实(准确)的3D估计；(2)这些估计是随机的，其概率分布由视觉系统编码；(3)线索整合过程导致3D结构的单一编码。相反，内在约束模型预测，线索估计是有偏见的、确定性的，线索整合导致了3D结构的两种不同编码。 使用最先进的视觉显示和运动跟踪设备，允许心理物理和精神运动反应测量，研究人员正在进行一系列全面的实验，旨在批判性地测试两种理论(贝叶斯和IC)。第一个工作包确定了哪种理论更好地解释了基于单一或组合深度线索的3D感知。第二个工作包确定深度线索是否如贝叶斯理论所建议的那样提供随机估计，或者线索是否如IC模型所提议的那样提供确定性噪声，而3D估计中的噪声是由外部实验因素引起的。第三个工作包展示了三维主观体验的差异如何与潜在的运动指导的3D编码的效率有关，但与潜在的感知判断无关。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Despite at least a century of scientific investigation, it is still not entirely understood how the human brain constructs a perception of three-dimensional (3D) objects and space from the 2D information reaching the eye through light rays.  It has been known for a long time that this ability is attained on the basis of a variety of different visual signals (that the brain interprets) called “depth cues”, which the brain then combines together to derive the 3D structure of the scene.  Examples of depth cues can be observed in paintings where linear perspective, shading, and even simple contours depict a 3D world on the flat canvas just as they do on the flat human retinas while people observe real 3D objects. However, how these cues encode 3D information and how the different cues are combined to generate our stable coherent perception of 3D visual space and objects remains poorly understood.  The prevailing theory in the scientific literature (Bayesian probabilistic inference theory) postulates that the brain derives a 3D structure of the world by determining how likely a particular 3D structure is given the information on the retina. Implementing this model requires a host of assumptions, such as that depth cues deliver “noisy” estimates of 3D parameters that are still, on average, accurate. However, a number of common and important observations cannot be fully explained by the Bayesian model and cast doubt on the critical computational assumptions of the model. Moreover, the Bayesian model struggles to explain important differences in the “quality” of 3-dimensionality that we perceive between viewing the real world and artificial situations such as pictorial images, or virtual or augmented reality (VR, AR). This project tests a new theory (the Intrinsic Constrained theory) that makes an entirely different and simpler set of assumptions compared to the Bayesian theory, but that can predict a wider range of perceptual phenomena.  The project integrates the IC theory with a recently proposed theory that postulates that the visual system does not generate a single encoding of 3D space, but two distinct encodings, one which is relevant to understanding the scene (i.e., 3D object shape and layout) and the other that underlies visually guided movements like reaching for and grasping an object.  The latter encoding is claimed to underlie the special subjective experience of 3-dimensionality that is most obvious while viewing stereoscopic images (e.g., 3D movies). This project will show that the IC model can efficiently incorporate the claims of two distinct representation of 3D space. In doing so, it is able to provide a better explanation of a range of fundamental aspects of our perception of 3-dimensionality that are challenging to explain with the prevailing model, including those required for a better understanding of factors important for developing 3D technology (e.g., VR and AR). This award supports empirical research that tests a computational model arising from the Intrinsic Constraint theory of cue integration against the prevailing computational model belonging to the Bayesian framework. Specifically, the new model challenges three main assumptions of the Bayesian model that will be tested by experiments in distinct work packages: (1) that depth cues on average provide veridical (accurate) 3D estimates; (2) that these estimates are stochastic and that their probability distributions are encoded by the visual system; and (3) that the process of cue integration results in a single encoding of 3D structure. Instead, the Intrinsic Constraint model predicts that cue estimates are biased, deterministic, and that cue integration results in two distinct encodings of 3D structure. Using state-of-the-art visual display and motion tracking apparatus that allow both psychophysical and psychomotor response measurements, the investigators are conducting a comprehensive set of experiments aimed at critically testing two theories (Bayesian and IC). The first work package establishes which theory better explains 3D perception based on single or combined depth cues. The second work package establishes if depth cues provide stochastic estimates as proposed by the Bayesian theory or if cues provide deterministic noise as proposed by the IC model, with noise in 3D estimates due to extraneous experimental factors. The third work package shows how differences in the subjective experience of 3-dimensionality is linked to the efficacy of the 3D encoding underlying guidance of movement, but not that underlying perceptual judgements.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.