Investigating Auditory and Audio-Visual Motion Perception in Real and Virtual 3D-Spaces as well as Effects of Attention and Training

现代工业工作环境的特点是自动化水平不断提高，自主行动的机器人与人类工人共享工厂车间。由于移动机器人构成了一种职业危害，因此深入了解人类对移动物体的感知以及与移动物体的互动非常重要。当移动的物体不在人类的视野中时，尤其如此。在这里，现代VR技术提供了模拟和评估安全高效工作流程的工作环境的可能性。为了比较人类在真实的和虚拟环境中的表现，我们不仅要分析行为测量，还要分析潜在神经认知过程的EEG相关性以及主观测量（例如，量化VR环境的可行性或任务的感知难度）。因此，该项目建立在DFG优先项目“AUDICTIVE”第一阶段的项目成果之上，在该项目中，开发并验证了一个基于VR的视听测试环境，用于调查三维空间中的空间感知和注意力。第二阶段的重点是在真实的和虚拟环境中的模拟工作场景中探索运动感知，以及在VR中验证视听和听觉运动感知。该项目将把动态组件集成到第一阶段的两个环境中，并进行一系列实验来研究运动感知。鉴于以前的研究结果，熟悉的作用与室内声学和注意的后果对多个移动声源的感知将进行调查。将使用双任务范式，其中参与者的精神负荷增加，通过第二个独立的任务，在移动物体的存在。最后，考虑到可能使用VR技术来训练真实世界的动作序列，将研究视听训练和学习对移动听觉对象的感知的影响，并推导出环境之间可能的转移。将包括年轻人和老年人，以确定可能的年龄相关影响。最后，我们将使用开发的VR平台来研究双耳语音增强算法（例如，用于去混响）并评估它们对空间感知的影响。因此，该项目将为视听运动的神经认知处理提供新的见解，并将通过使用VR技术为提高职业安全铺平道路。

Modern industrial work environments are characterized by increasing levels of automation and autonomously acting robots sharing the factory floor with human workers. As moving robots constitute an occupational hazard, a thorough understanding of human perception of and interaction with moving objects is of great importance. This is especially true when moving objects are not in the human's field of view. Here, modern VR technology offers the possibility to simulate and evaluate work environments with respect to safe and efficient workflows. To compare human performance in real and virtual environments, we will analyze not only behavioral measures but also EEG correlates of the underlying neurocognitive processes as well as subjective measures (e.g., quantifying the plausibility of the VR environment or the perceived difficulty of a task). Thus, this project builds on results of our project within the first phase of the DFG priority program "AUDICTIVE", in which a VR-based audio-visual test environment for investigating spatial perception and attention in three-dimensional space was developed and validated. The focus of this second phase is on the exploration of motion perception in a simulated work scenario in real and virtual environments and the validation of audio-visual and auditory motion perception in VR. The project will integrate dynamic components into both environments of the first phase and perform a series of experiments to study motion perception. In view of previous findings, the role of familiarization with the room acoustics and the consequences of attention on the perception of multiple moving sound sources will be investigated. A double-task paradigm will be used in which the participants’ mental load is increased through a second independent task in the presence of moving objects. Finally, considering a possible use of VR technology for training real-world action sequences, the effect of audio-visual training and learning on the perception of moving auditory objects will be studied and possible transfers between environments will be derived. Young and older people will be included to determine possible age-related influences. Finally, we will use the developed VR platform to investigate binaural speech enhancement algorithms (e.g., for dereverberation) and evaluate their impact on spatial perception. Thus, this project will provide novel insights into neural-cognitive processing of audio-visual motion and will pave the way to higher occupational safety by the use of VR technology.