Investigating Auditory and Audio-Visual Motion Perception in Real and Virtual 3D-Spaces as well as Effects of Attention and Training

现代工业工作环境的特点是自动化水平不断提高，自主行动的机器人与人类工人共享工厂车间。由于移动机器人构成了一种职业危害，彻底了解人类对移动对象的感知和相互作用是非常重要的。当移动的物体不在人类的视野内时，这一点尤其正确。在这里，现代VR技术提供了模拟和评估安全高效工作流程方面的工作环境的可能性。为了比较人类在真实和虚拟环境中的表现，我们不仅将分析行为测量，还将分析潜在神经认知过程的脑电相关性以及主观测量(例如，量化VR环境的可信程度或任务的感知难度)。因此，本项目建立在DFG优先项目AUDICTIVE第一阶段的成果基础上，在该项目中，开发并验证了基于VR的三维空间知觉和注意的视听测试环境。第二阶段的重点是探索在真实和虚拟环境中模拟工作场景中的运动感知，以及在VR中验证视听和听觉运动感知。该项目将在第一阶段的两个环境中集成动态组件，并进行一系列实验来研究运动感知。鉴于之前的研究结果，将调查熟悉房间声学的作用以及注意对多个运动声源的感知的影响。将使用双任务范式，在存在移动对象的情况下，通过第二个独立任务来增加参与者的精神负荷。最后，考虑到VR技术可能用于训练真实世界的动作序列，将研究视听训练和学习对运动听觉对象感知的影响，并推导出环境之间的可能转移。年轻人和老年人将被包括在内，以确定可能的年龄相关影响。最后，我们将使用开发的VR平台来研究双耳语音增强算法(例如，去混响)，并评估它们对空间感知的影响。因此，该项目将为视听运动的神经认知处理提供新的见解，并将为使用VR技术实现更高的职业安全铺平道路。

Modern industrial work environments are characterized by increasing levels of automation and autonomously acting robots sharing the factory floor with human workers. As moving robots constitute an occupational hazard, a thorough understanding of human perception of and interaction with moving objects is of great importance. This is especially true when moving objects are not in the human's field of view. Here, modern VR technology offers the possibility to simulate and evaluate work environments with respect to safe and efficient workflows. To compare human performance in real and virtual environments, we will analyze not only behavioral measures but also EEG correlates of the underlying neurocognitive processes as well as subjective measures (e.g., quantifying the plausibility of the VR environment or the perceived difficulty of a task). Thus, this project builds on results of our project within the first phase of the DFG priority program "AUDICTIVE", in which a VR-based audio-visual test environment for investigating spatial perception and attention in three-dimensional space was developed and validated. The focus of this second phase is on the exploration of motion perception in a simulated work scenario in real and virtual environments and the validation of audio-visual and auditory motion perception in VR. The project will integrate dynamic components into both environments of the first phase and perform a series of experiments to study motion perception. In view of previous findings, the role of familiarization with the room acoustics and the consequences of attention on the perception of multiple moving sound sources will be investigated. A double-task paradigm will be used in which the participants’ mental load is increased through a second independent task in the presence of moving objects. Finally, considering a possible use of VR technology for training real-world action sequences, the effect of audio-visual training and learning on the perception of moving auditory objects will be studied and possible transfers between environments will be derived. Young and older people will be included to determine possible age-related influences. Finally, we will use the developed VR platform to investigate binaural speech enhancement algorithms (e.g., for dereverberation) and evaluate their impact on spatial perception. Thus, this project will provide novel insights into neural-cognitive processing of audio-visual motion and will pave the way to higher occupational safety by the use of VR technology.