APlausE-MR - Audiovisual Plausibility and Experience in Multi-Party Mixed Reality

APlausE-MR侧重于分布式多方混合现实（MR）场景，这些场景涉及多组并发和远程用户。我们的假设是，现实的视听虚拟环境主要有助于产生地点幻觉，即身临其境的感觉，但要创造和维持这种逼真的幻觉，即对正在发生的事情的感觉是真实的，也需要虚拟形象的表现，即看起来、行为和听起来都是可信的。在这里，基于3d视频捕获的参与者表示的使用将与经典的化身类型表示进行比较。此外，支持用户与虚拟环境以及彼此之间的自然交互，通过共同感知提升体验的可信度。这些因素相互加强了可信性，甚至可以在不完美的情况下相互弥补。为了验证这一假设，我们提出的研究解决了三个主要挑战：(1)发展和整合视听技术，使虚拟世界中的群体内部和群体之间能够进行合理的交流；(2)识别和确定影响多方虚拟现实中视听体验合理性的因素；(3)开发适合的评估方法和技术，用于分析配置和分布的视听虚拟现实中的用户和群体行为。该项目的博士生将得到三个研究小组的支持，他们在多用户虚拟现实，沉浸式远程呈现，高度逼真的空间音频体验以及视听环境和多方通信体验质量评估方面具有重要的专业知识。

APlausE-MR focuses on distributed multi-party mixed reality (MR) scenarios that involve groups of collocated and remote users. It is our hypothesis that realistic audiovisual virtual environments contribute mainly to the place illusion, the sense of being there, but that creating and maintaining the illusion of plausibility, the sensation of what is happening is real, also requires avatar representations that look, behave and sound credible. Here, the usage of participant representations based on 3D-video-capture will be compared to classic avatar-type representations. Furthermore, supporting the users’ natural interaction with the virtual environment and with each other promotes the credibility of the experience through joint perception. These factors mutually reinforce plausibility and can even compensate each other in case of imperfections. To validate this hypothesis our proposed research addresses three main challenges: (1) The development and integration of the audiovisual technologies that enable plausible communication within a group and between groups in virtual worlds, (2) the identification and qualification of factors influencing the plausibility of audiovisual experiences in multi-party IVEs and (3) the development of suitable evaluation methods and technologies for the analysis of user and group behavior in collocated and distributed audiovisual IVEs. The PhD students of this project will be supported by three research groups with significant expertise in multi-user virtual reality, immersive telepresence, highly realistic spatial audio experiences, and the assessment of the quality of experience in audiovisual environments and for multi-party communication.