EPSRC-SFI: SpheryStream

EPSRC-SFI：SpheryStream

• 批准号: EP/T03324X/1
• 负责人: Laura TONI
• 金额: $ 32.81万
• 依托单位: University College London
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2021
• 资助国家: 英国
• 起止时间: 2021 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FT03324X%2F1
• 关键词: EPSRC; SFI; SpheryStream

This proposal addresses the important and not well-studied area in the fast-developing engineering space of the virtual reality and immersive technology. Our final vision is to endow any final user with an unprecedented sense of full immersion in virtual environments for any applicative scenario and with any connectivity level. This requires VR systems that operate at scale, in a personalized manner, remaining bandwidth-tolerant whilst meeting quality and latency criteria. This can be accomplished only by a fundamental revolution of the coding/streaming/rendering chain that has to put the interactive user at the heart of the system rather than at the end of the chain.Specifically, bringing our final vision to reality requires constant innovation in VR streaming technology to overcome the following main challenges: i) the spherical format of the video content in a scenario in which media codecs are optimized for planar content; ii) the interactive behaviour of users that introduces uncertainty on the popularity of the video content. To overcome the former challenges, the spherical video is currently projected on a bidimensional planar domain, but this introduces noticeable deformation since the sphere is not a developable surface. To address the uncertainty of users' behaviours, predicting models have been investigated recently, but none of them actually work on the spherical (not deformed) domain. To overcome the above challenges in immersive technology, there is the need to develop an efficient tool for navigation patterns analysis in the spherical domain and leverage on that to predict users' behaviour and build the entire coding-delivery-rendering pipeline as a user- and geometry-centric system. This proposal focuses on develop novel VR delivery strategies, which will rely on the analysis and prediction of the navigation patterns of the omnidirectional content navigation, in a user, content- and application-dependent fashion. Predicting users' behaviour will allow to delivery only the content that will be actually consumed by the users, minimising the transmission resources consumption while maximising the user's Quality of Experience. The key novelty will be to address the above challenges working directly on the spherical domain, rather than on the projected (thus deformed) bidimensional domain. The developed VR system will be tested via a real omnidirectional video streaming testbed that will be implemented within the scope of the project. The outcomes of the project will be three-folds: i) user- content- and application-dependent VR streaming platform, optimized from the source codec to the delivery platform; ii) users analysis tools able to assess users' behaviour (when interactive with the content) in an objective fashion to identify common navigation patterns, and to predict future behaviours; iii) VR streaming testbed to test the proposed technology.

该提案解决了虚拟现实和沉浸式技术快速发展的工程领域中重要而未得到充分研究的领域。我们的最终愿景是赋予任何最终用户一种前所未有的完全沉浸在虚拟环境中的感觉，适用于任何应用场景和任何连接级别。这需要VR系统以个性化的方式大规模运行，保持带宽容忍，同时满足质量和延迟标准。要实现这一点，必须对编码/流媒体/渲染链进行根本性的变革，将交互用户置于系统的核心，而不是链的末端。具体而言，要实现我们的最终愿景，需要不断创新VR流媒体技术，以克服以下主要挑战：i）在媒体编解码器针对平面内容被优化的情况下视频内容的球形格式; ii）用户的交互行为，其引入了视频内容的流行度的不确定性。为了克服前面的挑战，球形视频目前被投影在二维平面域上，但是这引入了明显的变形，因为球体不是可展表面。为了解决用户行为的不确定性，最近已经研究了预测模型，但它们实际上都不适用于球形（非变形）域。为了克服沉浸式技术中的上述挑战，需要开发一种用于球形域中的导航模式分析的有效工具，并利用该工具来预测用户的行为，并将整个编码-交付-渲染管道构建为以用户和几何为中心的系统。这项提案的重点是开发新的VR交付策略，这将依赖于分析和预测的导航模式的全向内容导航，在用户，内容和应用程序依赖的方式。预测用户的行为将允许仅递送将由用户实际消费的内容，从而最小化传输资源消耗，同时最大化用户的体验质量。关键的新奇将是直接在球形域上工作，而不是在投影（因此变形）的二维域上解决上述挑战。开发的VR系统将通过一个真实的全向视频流测试平台进行测试，该平台将在项目范围内实施。该项目的成果将有三个方面：i）依赖于用户内容和应用程序的VR流媒体平台，从源编解码器到交付平台进行了优化; ii）用户分析工具，能够以客观的方式评估用户的行为（当与内容交互时），以识别常见的导航模式，并预测未来的行为; iii）VR流媒体测试平台，以测试拟议的技术。

项目成果

Correlation between Entropy and Prediction Error in VR Head Motion Trajectories

• DOI: 10.1145/3607546.3616805
• 发表时间: 2023-10
• 期刊: Proceedings of the 2nd International Workshop on Interactive eXtended Reality
• 作者: Silvia Rossi;Laura Toni;Pablo César

Immersive Video Technologies

沉浸式视频技术

• DOI: 10.1016/b978-0-32-391755-1.00009-2
• 发表时间: 2023
• 作者: Rossi S

Extending 3-DoF Metrics to Model User Behaviour Similarity in 6-DoF Immersive Applications

扩展 3-DoF 指标以对 6-DoF 沉浸式应用中的用户行为相似性进行建模

• DOI: 10.1145/3587819.3590976
• 发表时间: 2023
• 作者: Rossi S

A New Challenge: Behavioural Analysis Of 6-DOF User When Consuming Immersive Media

新挑战：六自由度用户消费沉浸式媒体时的行为分析

• DOI: 10.1109/icip42928.2021.9506525
• 发表时间: 2021
• 作者: Rossi S

Influence of Narrative Elements on User Behaviour in Photorealistic Social VR

真实感社交 VR 中叙事元素对用户行为的影响

• DOI: 10.1145/3458307.3463371
• 发表时间: 2021
• 作者: Rossi S