Mixed Reality Environment for Immersive Experience of Art and Culture

参与艺术和音乐可以对身心健康产生一系列积极的影响。然而，由于各种原因，获得音乐体验的许多好处可能会受到限制：(1)场地的地理位置，(2)进入方面的个人限制，(3)音乐会和表演的时间表限制，(4)体验的费用。由于这些限制，观看音乐会和演出的努力和成本超过了参与者从体验中获得的额外价值。在Xtreme项目中，我们通过开发以人为中心的混合现实(MR)解决方案来消除这些限制，该解决方案不要求参与者实际在场，而是通过MR技术将音乐会和表演带到远程位置，同时保留身临其境的体验，甚至通过整合真实和虚拟内容赋予其更多维度。我们的解决方案还允许与选定的用户社交圈一起提供虚拟共享的现实体验。该解决方案消除了地理限制，因为通过MR解决方案，几乎可以从任何地方访问音乐会和表演，因此对于访问受限的人来说，它们更具包容性。同样，提供全球接触艺术和音乐的途径为音乐家和艺术家提供了更广泛的受众，而不是通过实际场地的大小来限制座位数量。由于这种解决方案省去了去听音乐会和演出的必要，它还提供了一种更环保的选择。与当前最先进的解决方案不同，我们的解决方案旨在为演唱会和表演提供MR解决方案，这些演唱会和表演的3D视觉和声音都融合到了用户的房间中。我们还将通过重建用户及其在共享访问中的行为来提供最大限度的在线体验，并通过为声音提供额外的视觉层来增强沉浸感。

Engagement with the art and music can have a range of positive effects on physical and mental health. However, access to the many benefits of musical experience can be limited due to various reasons: (1) the geographic location of the venue, (2) personal limitations in access, (3) limitation on the schedule of the concerts and performances, (4) cost of the experience. Due to these limitations the effort and cost of getting access to the concerts and performances override additional value the participant would get from the experiences. In XTREME project, we remove these limitations by developing a human-centered mixed reality (MR) solution that, instead of requiring the participant to be physically present in the venue, brings the concerts and performances to a remote location via MR technology while retaining the immersive experience, and even giving additional dimensions to it by integrating real and virtual contents. Our solution also allows for virtually shared, realistic experience together with selected social circles of the user. The solution removes geographic limitations as by the MR solution the concerts and performances can be accessed virtually from anywhere and they hence are more inclusive for people with limited access. Likewise, providing a global access to the art and music provides a wider audience for musicians and artists, not limiting the number of seats by the size of the physical venue. As this solution removes the necessity to travel to concerts and performances, it also provides a greener alternative. In contrast to the current state-of-the-art, our solution targets at making an MR solution to the concerts and performances where those are in in 3D, both for visuals and sound, blended into the room of the users. We will also provide maximised experience of presence via reconstruction of the users and their behaviour in the shared access, and enhance the immersion by providing an additional visual layer to the sound.