Influence of audio rendering in virtual environments on realism, presence, and socio-cognitive processing

交互式虚拟环境(IVES)为研究人类在复杂场景中的感知和社会认知过程提供了独特的新可能性。在IVES中，生态上有效的环境条件可以与高度的实验控制相结合。尽管研究人员经常将音频作为一种方式，但高度逼真的音频集成仍然是一个例外。本项目的主要目标是确定有关双耳音频呈现的要求，以实现虚拟社会场景的高度逼真的视听呈现，并表征音频呈现对(社会)在场和社会认知过程的影响，如情绪、同理心和(社会)焦虑。在第一个融资阶段，重点是实现尽可能高的现实主义，在第二个融资阶段，我们将专注于在自我报告、行为和神经水平(通过脑电测量)上表征虚拟社交场景中自上而下效应和刺激相关自下而上效应之间的相互作用。这种互动将通过考虑和分析预期、情感、熟悉、社交压力和存在以及虚拟社交场景中关于这些场景中双耳渲染的不同复杂程度的更高级别的社会认知(例如，移情)来调查：阶段1中开发的(高现实主义)渲染是起点，它将扩展到多个语音和非语音观众席相关的来源以及更高级的自身语音渲染，以便能够实现接近真实生活的交互社交场景。将调查习惯化和训练，以及头部跟踪的替代方法，以促进视听IVE的使用，并使其能够进一步应用于临床，如治疗社交焦虑症。总体而言，该项目将有助于更好地理解心理和生理过程及其功能相关性，以提供真实生活环境的可信表现。

Interactive Virtual Environments (IVEs) provide unique new possibilities to study human perception and socio-cognitive processing in complex scenes. In IVEs, ecologically valid environmental conditions can be combined with a high degree of experimental control. Although researchers often included audio as a modality, the integration of highly realistic audio is still an exception. The overarching goal of this project is to identify the requirements regarding binaural audio rendering to achieve highly realistic audiovisual presentations of virtual social scenes and to characterize the effects of the audio rendering on (social) presence and socio-cognitive processes such as emotion, empathy, and (social) anxiety. Whereas in the first funding phase, the focus was on achieving the highest possible realism, in the second funding phase we will concentrate on characterizing the interaction between top-down effects and stimulus-related bottom-up effects in virtual social interaction scenarios on a self-report, behavioral, and on a neural level (measured via EEG). This interaction will be investigated by considering and analyzing expectations, emotion, familiarization, social stress, and presence as well as higher-order social cognition (e.g., empathy) in virtual social scenarios with different degrees of complexity regarding the binaural rendering in these scenarios: the (high realism) rendering developed in phase 1 is the starting point, which will be extended to multiple speech and non-speech auditorium-related sources as well as to a more advanced own-voice renderings in order to enable close-to-real-life interactive social scenes. Habituation and training, as well as alternatives to head-tracking, will be investigated to facilitate the use of audiovisual IVEs and to enable further clinical applications such as treating social anxiety disorders. Overall this project will contribute to better understand psychological and physiological processes and their functional relevance to provide plausible representations of real-life environments.