Adventures in mirror world: Uncovering the cognitive and sensory basis for natural behaviour in virtual reality

除了其明显的娱乐价值外，VR设备在对人类感知和行为的系统实验研究方面正变得越来越有吸引力，因为它们有望对接近自然环境复杂性的环境进行实验控制。如果我们在VR设备的帮助下调查行为或训练某些现实世界的任务(例如，在体育或医疗培训中)，我们希望我们获得的洞察力或技能会转移到现实世界的环境中。然而，目前还不清楚虚拟环境和自然环境中的行为是否以及在多大程度上是可比较的。自然环境和虚拟环境中行为差异的一个原因是，目前可用的VR设备模拟的环境在感知上没有自然环境那么复杂，并且缺乏许多我们在现实世界中用来计划和执行行动的信息来源(例如，缺失或不适当地实现了对深度的线索)。然而，目前可用的虚拟现实设备的感知限制可能并不是行为差异的唯一来源。到目前为止，人们在很大程度上忽略了认知因素如何影响虚拟现实中的行为。特别是，与虚拟对象互动的意识可以改变人们对我们行为后果的预期。例如，虽然我们对与真实对象的碰撞的直接后果(例如，对对象的疼痛或伤害和/或损坏)有明确的预期，但与虚拟对象的碰撞通常不具有这样的直接后果。因此，即使虚拟现实在感知上是完美的，只要人们认识到它的虚拟本质，认知效应可能会改变我们在虚拟现实中的行为方式(例如，我们移动得多么谨慎)。因此，了解这些认知效应的确切性质是至关重要的。许多VR应用程序的另一个可能影响我们行为的特征是，用户通常不直接与真实或虚拟的物体互动，而是使用某种设备(例如，操纵杆)来操纵它们。在这种情况下，生物力学作用力和由此产生的运动之间的关系不同于它们的自然关系。这一点至关重要，因为我们知道，自然动作是由生物力学需求和我们身体的生物力学约束决定的。系统地研究这些因素的影响是困难的，因为由于目前可用的VR设备的限制，不可能分离它们的影响。例如，减少的感知复杂性对运动的影响不能很容易地与与虚拟对象交互的知识所产生的影响分开。在这里，我们提出了一种方法，允许我们直接比较自然和虚拟环境中的行为，而不受当前可用的VR设备的限制。我们将使用一种功能强大但在这类研究中未得到充分利用的工具来模拟虚拟环境：镜子。镜子简单易用，但在视觉上却是最完整、最令人信服的虚拟现实设备。使用镜子模拟VR的优点是，镜像环境提供了与相应自然环境相同的视觉信息。因此，我们可以研究虚拟环境中的行为，同时能够控制降低感知复杂性的混杂效果，这是目前可用的VR设备不可能做到的。在这个模拟的虚拟现实中，我们将使用生态上有效的避障和抓取任务来系统地研究VR影响行为的三个特征(即降低感知复杂性、物体的虚拟和动作的虚拟)。我们的目标是为虚拟现实的实现提供科学依据，有助于研究人类行为，并促进对知觉、认知和视觉运动过程相互关系的了解。

Besides their obvious value for entertainment, VR devices are becoming increasingly attractive for the systematic experimental investigation of human perception and behaviour because they promise the possibility of experimental control over environments that approach the complexity of natural settings. If we investigate behaviour or train certain real-world tasks (e.g., in sports or medical training) with the help of VR devices, we expect that the insights we gain or the skills we acquire transfer to real-world settings. Yet, it is currently still unclear if and to what degree behaviour in virtual and natural environments is comparable.One reason for differences between behaviour in natural and virtual environments is that currently available VR devices simulate environments that are perceptually less complex than their natural counterparts and lack many sources of information that we use to plan and execute our actions in the real world (e.g., missing or inappropriately implemented cues to depth). However, perceptual limitations of currently available VR devices may not be the only source of differences in behaviour. What has so far largely been neglected is how cognitive factors shape behaviour in VR. Particularly, the awareness of interacting with virtual objects can change expectations about the consequence of our actions. For example, while we have clear expectations regarding the immediate consequences of collisions with real objects (e.g., pain or injury and/or damage to the object), collisions with virtual objects usually do not have such direct consequences. Thus, even if VR was perceptually perfect, as long as people are cognisant of its virtual nature, the cognitive effects are likely to alter the way we act in VR (e.g., how cautiously we move). It is therefore essential to understand the exact nature of these cognitive effects.Another feature of many VR applications that might affect our actions is that users often do not directly interact with real or virtual objects but instead use some device (e.g., a joystick) to manipulate them. In that case, the relation between biomechanical effort and resulting movement differs from their natural relation. This is crucial as we know that natural movements are shaped by biomechanical demands and the biomechanical constraints of our body.What makes the systematic investigation of the influence of these factors difficult is that due to the limitations of currently available VR devices, it is impossible to disentangle their effects. For example, the effects of the reduced perceptual complexity on movements cannot be easily separated from those resulting from the knowledge of interacting with virtual objects. Here, we propose a method that allows us to directly compare behaviour in natural and virtual environments independent of the limitations of currently available VR devices. We will simulate a virtual environment by using a powerful but in this kind of research underutilised tool: mirrors. Mirrors are simple and easy to use, yet they are visually the most complete and convincing virtual reality device available. Using mirrors to simulate VR has the advantage that the mirrored environment provides the same visual information as the corresponding natural environment. Therefore, we can investigate behaviour in virtual environments while being able to control the confounding effects of reduced perceptual complexity which is impossible with currently available VR devices. With this simulated virtual reality, we will systematically investigate the three features of VR influencing behaviour (i.e., reduction of perceptual complexity, virtuality of objects and virtuality of movements) using ecologically valid obstacle avoidance and grasping tasks. Our goal is to provide the scientific basis for implementations of VR that are useful for the investigation of human actions and advance knowledge on the interrelationship of perceptual, cognitive and visuomotor processes.