Adventures in mirror world: Uncovering the cognitive and sensory basis for natural behaviour in virtual reality

镜子世界的冒险：揭示虚拟现实中自然行为的认知和感官基础

• 批准号: ES/V005170/1
• 负责人: Constanze Hesse
• 金额: $ 76.25万
• 依托单位: University of Aberdeen
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2021
• 资助国家: 英国
• 起止时间: 2021 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=ES%2FV005170%2F1
• 关键词: Adventures; mirror; world; Uncovering; cognitive

Besides their obvious value for entertainment, VR devices are becoming increasingly attractive for the systematic experimental investigation of human perception and behaviour because they promise the possibility of experimental control over environments that approach the complexity of natural settings. If we investigate behaviour or train certain real-world tasks (e.g., in sports or medical training) with the help of VR devices, we expect that the insights we gain or the skills we acquire transfer to real-world settings. Yet, it is currently still unclear if and to what degree behaviour in virtual and natural environments is comparable.One reason for differences between behaviour in natural and virtual environments is that currently available VR devices simulate environments that are perceptually less complex than their natural counterparts and lack many sources of information that we use to plan and execute our actions in the real world (e.g., missing or inappropriately implemented cues to depth). However, perceptual limitations of currently available VR devices may not be the only source of differences in behaviour. What has so far largely been neglected is how cognitive factors shape behaviour in VR. Particularly, the awareness of interacting with virtual objects can change expectations about the consequence of our actions. For example, while we have clear expectations regarding the immediate consequences of collisions with real objects (e.g., pain or injury and/or damage to the object), collisions with virtual objects usually do not have such direct consequences. Thus, even if VR was perceptually perfect, as long as people are cognisant of its virtual nature, the cognitive effects are likely to alter the way we act in VR (e.g., how cautiously we move). It is therefore essential to understand the exact nature of these cognitive effects.Another feature of many VR applications that might affect our actions is that users often do not directly interact with real or virtual objects but instead use some device (e.g., a joystick) to manipulate them. In that case, the relation between biomechanical effort and resulting movement differs from their natural relation. This is crucial as we know that natural movements are shaped by biomechanical demands and the biomechanical constraints of our body.What makes the systematic investigation of the influence of these factors difficult is that due to the limitations of currently available VR devices, it is impossible to disentangle their effects. For example, the effects of the reduced perceptual complexity on movements cannot be easily separated from those resulting from the knowledge of interacting with virtual objects. Here, we propose a method that allows us to directly compare behaviour in natural and virtual environments independent of the limitations of currently available VR devices. We will simulate a virtual environment by using a powerful but in this kind of research underutilised tool: mirrors. Mirrors are simple and easy to use, yet they are visually the most complete and convincing virtual reality device available. Using mirrors to simulate VR has the advantage that the mirrored environment provides the same visual information as the corresponding natural environment. Therefore, we can investigate behaviour in virtual environments while being able to control the confounding effects of reduced perceptual complexity which is impossible with currently available VR devices. With this simulated virtual reality, we will systematically investigate the three features of VR influencing behaviour (i.e., reduction of perceptual complexity, virtuality of objects and virtuality of movements) using ecologically valid obstacle avoidance and grasping tasks. Our goal is to provide the scientific basis for implementations of VR that are useful for the investigation of human actions and advance knowledge on the interrelationship of perceptual, cognitive and visuomotor processes.

除了具有明显的娱乐价值外，VR设备对于人类感知和行为的系统实验研究也越来越有吸引力，因为它们有望对接近自然环境复杂性的环境进行实验控制。如果我们在VR设备的帮助下调查行为或训练某些现实世界的任务（例如，在体育或医疗培训中），我们希望我们获得的见解或获得的技能能够转移到现实世界的设置中。然而，目前尚不清楚虚拟环境和自然环境中的行为是否具有可比性，以及在多大程度上具有可比性。自然环境和虚拟环境中的行为存在差异的一个原因是，目前可用的VR设备模拟的环境在感知上没有自然环境那么复杂，并且缺乏我们在现实世界中用来计划和执行行动的许多信息来源（例如，缺失或不恰当地执行深度线索）。然而，目前可用的VR设备的感知限制可能不是行为差异的唯一来源。到目前为止，很大程度上被忽视的是认知因素如何影响VR中的行为。特别是，与虚拟对象交互的意识可以改变我们对行为后果的预期。例如，虽然我们对与真实物体碰撞的直接后果（例如，疼痛或伤害和/或损坏物体）有明确的期望，但与虚拟物体碰撞通常不会产生这种直接后果。因此，即使VR在感知上是完美的，只要人们认识到它的虚拟本质，认知效应就有可能改变我们在VR中的行为方式（例如，我们移动的小心程度）。因此，有必要了解这些认知效应的确切性质。许多VR应用程序可能会影响我们的行为的另一个特点是，用户通常不会直接与真实或虚拟对象交互，而是使用一些设备（例如操纵杆）来操纵它们。在这种情况下，生物力学努力和由此产生的运动之间的关系不同于它们的自然关系。这是至关重要的，因为我们知道自然运动是由我们身体的生物力学需求和生物力学约束决定的。对这些因素的影响进行系统调查的困难之处在于，由于目前可用的VR设备的局限性，无法理清它们的影响。例如，降低感知复杂性对运动的影响不能轻易地与那些与虚拟对象交互的知识所产生的影响分开。在这里，我们提出了一种方法，使我们能够直接比较自然和虚拟环境中的行为，而不受当前可用VR设备的限制。我们将模拟一个虚拟的环境，通过使用一个强大的，但在这类研究中未充分利用的工具：镜子。镜子简单易用，但它们是视觉上最完整、最令人信服的虚拟现实设备。使用镜像模拟VR的优点是镜像环境提供了与相应的自然环境相同的视觉信息。因此，我们可以研究虚拟环境中的行为，同时能够控制降低感知复杂性的混淆效应，这是目前可用的VR设备无法实现的。有了这个模拟的虚拟现实，我们将系统地研究VR影响行为的三个特征（即，减少感知复杂性，物体的虚拟性和运动的虚拟性），使用生态有效的避障和抓取任务。我们的目标是为VR的实现提供科学基础，这对于研究人类行为和推进感知、认知和视觉运动过程的相互关系是有用的。