Vision in Virtual Reality

“Ceci n'est pas une pipe”（这不是一个烟斗）是ren<s:1> Magritte在他1929年的画作“La trahison des images”中对烟斗的描绘下写的。长期以来，艺术家和哲学家们一直在研究图像空间和视觉空间之间的差异。图画空间是我们在看一幅画、在电脑显示器上或在电影院的屏幕上看到的空间。相比之下，视觉空间是当我们睁开眼睛看向我们面前的世界时所看到的。这也是我们在精心设计的虚拟现实中所体验到的。体验上的差异是巨大的。当我们看一幅画的时候，我们可以理解所描绘的空间和它所包含的物体之间的空间关系。但我们自己不是那个空间的一部分。相反，在现实世界中，我们的身体在我们面前打开的视觉空间中占据了一个确定的位置。我们对“在场”的体验会对我们如何遇到物体，尤其是其他人产生重大影响。事实上，我们的视觉系统似乎以两种完全不同的模式运作，这取决于它是处理视觉空间还是处理图像空间。我使用术语“在场模式”和“图片模式”来指代这两种模式。我们是否处于在场模式似乎只在很小程度上取决于认知控制的因素，而是由感知因素决定的。***我的目标是描述(a)决定我们处于哪种模式的感知因素，以及(b)视觉模式具有哪些感知、认知和生理后果。***我的方法很大程度上基于头戴式虚拟现实（VR）技术的使用。VR允许我们建立视觉空间。与现实世界一样，这些视觉空间也可以包含图像空间。例如，我们可以在虚拟客厅的墙上挂一幅画，或者我们可以在虚拟桌子上放一个电脑显示器。通过完全控制头戴式显示器提供的刺激，我们可以识别刺激属性，以确定它是在存在模式还是在图像模式下被解释。***使用这项技术，我还将研究视觉模式如何影响他人的感知，以及它如何影响人们对可怕、紧张情况的反应。我预计这些影响是巨大的。一个年轻人是在屏幕上玩暴力电脑游戏，还是在虚拟现实中玩暴力游戏，对他们的认知和情感发展可能会产生非常不同的影响。成功地训练医学生应对紧急情况，或成功地通过暴露疗法减轻焦虑症患者的特定恐惧症，可能也在很大程度上取决于我们对虚拟现实视觉的理解。***快速发展的虚拟现实行业需要精通虚拟现实硬件和软件的专家，他们了解人类视觉系统的内部，并且是经验丰富的实验家。我正在这三个方面训练我的学生。

"Ceci n'est pas une pipe" (this is not a pipe) writes René Magritte right under the depiction of a pipe in his 1929 painting “La trahison des images”. Artists and philosophers have long theorized about the differences between pictorial and visual spaces. Pictorial spaces are the ones we see when we look at a picture, onto a computer monitor or at the screen of a movie theatre. Visual space, in contrast, is what we see when we open our eyes to look out into the world in front of us. It is also what we experience in well-designed virtual realities.***The experiential difference is huge. When looking at a picture, we can understand the depicted space and spatial relations between the objects it contains. But we are not ourselves part of that space. In the real world, in contrast, our body occupies a defined location in the visual space that opens in front of us. We experience “presence” with significant consequences on how we encounter objects and particularly other people. It seems, in fact, that our visual system operates in two entirely different modes depending on whether it is dealing with visual space or with pictorial spaces. I use the terms “presence mode” and “picture mode” to refer to these two modes. Whether or not we are in presence mode seems to depend only to a very minor extent on factors that are under cognitive control, but is rather determined by perceptual factors. ***My goal is to characterize (a) the perceptual factors that determine which mode we are in, and (b) what perceptual, cognitive, and physiological consequences the visual mode has.***My methodology is heavily based on the use of head-mounted virtual reality (VR) technology. VR allows us to set up visual spaces. As in the real world, these visual spaces can also contain pictorial spaces. For instance, we can hang a picture on the wall of a virtual living room, or we can place a computer monitor on a virtual desk. Having full control over the stimuli delivered by the head-mounted display, we can identify the stimulus properties that determine whether it is interpreted in presence mode or in picture mode.***Using that technology, I will also look into how the visual mode affects the perception of other people, and how it influences people's responses to frightening, stressful situations. I expect that these effects are huge. Whether a young person plays a violent computer game on a screen or in VR might have very different impacts on their cognitive and emotional development. The success of training a medical student to cope with emergency situations, or of exposure therapy to alleviate a specific phobia in an anxiety patient probably also depends heavily on our understanding of vision in virtual reality.***The quickly expanding industries around virtual realities need experts who are at the same time fluent with the hardware and software involved in VR, who understand the human visual system inside out, and who are seasoned experimenters. I am training my students in all three areas.