A test of a novel non-probabilistic model of 3D cue integration

3D 线索整合的新型非概率模型的测试

• 批准号: 2120610
• 负责人: Fulvio Domini
• 金额: $ 50万
• 依托单位: Brown University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2120610&HistoricalAwards=false
• 关键词: test; novel; non; probabilistic; model

Despite at least a century of scientific investigation, it is still not entirely understood how the human brain constructs a perception of three-dimensional (3D) objects and space from the 2D information reaching the eye through light rays.  It has been known for a long time that this ability is attained on the basis of a variety of different visual signals (that the brain interprets) called “depth cues”, which the brain then combines together to derive the 3D structure of the scene.  Examples of depth cues can be observed in paintings where linear perspective, shading, and even simple contours depict a 3D world on the flat canvas just as they do on the flat human retinas while people observe real 3D objects. However, how these cues encode 3D information and how the different cues are combined to generate our stable coherent perception of 3D visual space and objects remains poorly understood.  The prevailing theory in the scientific literature (Bayesian probabilistic inference theory) postulates that the brain derives a 3D structure of the world by determining how likely a particular 3D structure is given the information on the retina. Implementing this model requires a host of assumptions, such as that depth cues deliver “noisy” estimates of 3D parameters that are still, on average, accurate. However, a number of common and important observations cannot be fully explained by the Bayesian model and cast doubt on the critical computational assumptions of the model. Moreover, the Bayesian model struggles to explain important differences in the “quality” of 3-dimensionality that we perceive between viewing the real world and artificial situations such as pictorial images, or virtual or augmented reality (VR, AR). This project tests a new theory (the Intrinsic Constrained theory) that makes an entirely different and simpler set of assumptions compared to the Bayesian theory, but that can predict a wider range of perceptual phenomena.  The project integrates the IC theory with a recently proposed theory that postulates that the visual system does not generate a single encoding of 3D space, but two distinct encodings, one which is relevant to understanding the scene (i.e., 3D object shape and layout) and the other that underlies visually guided movements like reaching for and grasping an object.  The latter encoding is claimed to underlie the special subjective experience of 3-dimensionality that is most obvious while viewing stereoscopic images (e.g., 3D movies). This project will show that the IC model can efficiently incorporate the claims of two distinct representation of 3D space. In doing so, it is able to provide a better explanation of a range of fundamental aspects of our perception of 3-dimensionality that are challenging to explain with the prevailing model, including those required for a better understanding of factors important for developing 3D technology (e.g., VR and AR). This award supports empirical research that tests a computational model arising from the Intrinsic Constraint theory of cue integration against the prevailing computational model belonging to the Bayesian framework. Specifically, the new model challenges three main assumptions of the Bayesian model that will be tested by experiments in distinct work packages: (1) that depth cues on average provide veridical (accurate) 3D estimates; (2) that these estimates are stochastic and that their probability distributions are encoded by the visual system; and (3) that the process of cue integration results in a single encoding of 3D structure. Instead, the Intrinsic Constraint model predicts that cue estimates are biased, deterministic, and that cue integration results in two distinct encodings of 3D structure. Using state-of-the-art visual display and motion tracking apparatus that allow both psychophysical and psychomotor response measurements, the investigators are conducting a comprehensive set of experiments aimed at critically testing two theories (Bayesian and IC). The first work package establishes which theory better explains 3D perception based on single or combined depth cues. The second work package establishes if depth cues provide stochastic estimates as proposed by the Bayesian theory or if cues provide deterministic noise as proposed by the IC model, with noise in 3D estimates due to extraneous experimental factors. The third work package shows how differences in the subjective experience of 3-dimensionality is linked to the efficacy of the 3D encoding underlying guidance of movement, but not that underlying perceptual judgements.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

尽管经过了至少一个世纪的科学研究，人们仍然没有完全理解人类大脑是如何从通过光线到达眼睛的二维信息中构建对三维物体和空间的感知的。  我们早就知道，这种能力是建立在各种不同的视觉信号（大脑解释）的基础上的，这些信号被称为“深度线索”，然后大脑将它们组合在一起，得出场景的3D结构。  深度线索的例子可以在绘画中观察到，其中线性透视，阴影，甚至简单的轮廓在平面画布上描绘3D世界，就像人们在观察真正的3D物体时在平面人类视网膜上所做的那样。然而，这些线索是如何编码3D信息的，以及不同的线索是如何结合起来产生我们对3D视觉空间和物体的稳定连贯感知的，我们仍然知之甚少。  科学文献中流行的理论（贝叶斯概率推断理论）假设大脑通过确定特定3D结构在视网膜上提供信息的可能性来获得世界的3D结构。实现这个模型需要大量的假设，比如深度线索提供的3D参数的“噪声”估计，平均来说，仍然是准确的。然而，一些常见的和重要的观测结果不能被贝叶斯模型完全解释，并对该模型的关键计算假设产生了怀疑。此外，贝叶斯模型努力解释我们在观看真实世界和人工场景（如图像、虚拟现实或增强现实（VR， AR））之间感知到的三维“质量”的重要差异。这个项目测试了一种新的理论（内在约束理论），与贝叶斯理论相比，它提出了一套完全不同的、更简单的假设，但它可以预测更广泛的感知现象。  该项目将IC理论与最近提出的理论相结合，该理论假设视觉系统不会产生3D空间的单一编码，而是产生两种不同的编码，一种与理解场景（即3D物体形状和布局）相关，另一种是视觉引导运动的基础，如伸手抓住物体。  后一种编码被认为是特殊的三维主观体验的基础，这种体验在观看立体图像（如3D电影）时最为明显。该项目将表明，IC模型可以有效地结合两种不同的3D空间表示。通过这样做，它能够更好地解释我们对三维感知的一系列基本方面，这些方面很难用主流模型来解释，包括那些需要更好地理解开发3D技术（例如，VR和AR）的重要因素的方面。该奖项支持实证研究，该研究测试了由线索整合的内在约束理论产生的计算模型，而不是属于贝叶斯框架的主流计算模型。具体来说，新模型挑战了贝叶斯模型的三个主要假设，这些假设将通过不同工作包的实验进行测试： (1)深度线索平均提供真实（准确）的3D估计；(2)这些估计是随机的，它们的概率分布是由视觉系统编码的；(3)线索整合过程导致三维结构的单一编码。相反，内在约束模型预测线索估计是有偏见的，确定性的，线索整合导致两种不同的3D结构编码。 使用最先进的视觉显示和运动跟踪设备，允许心理物理和心理运动反应测量，研究人员正在进行一套全面的实验，旨在严格测试两种理论（贝叶斯和IC）。第一个工作包确定了哪个理论更好地解释了基于单个或组合深度线索的3D感知。第二个工作包确定深度线索是否提供贝叶斯理论提出的随机估计，或者线索是否提供IC模型提出的确定性噪声，其中3D估计中的噪声是由于无关的实验因素。第三个工作包显示了三维主观体验的差异是如何与运动指导下的3D编码的有效性联系在一起的，而不是与潜在的感知判断联系在一起的。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

On Human-like Biases in Convolutional Neural Networks for the Perception of Slant from Texture

• DOI: 10.1145/3613451
• 发表时间: 2023-08
• 期刊: ACM Transactions on Applied Perception
• 影响因子: 1.6
• 作者: Yuanhao Wang;Qian Zhang;Celine Aubuchon;Jovan T. Kemp;F. Domini;J. Tompkin