EPSRC-SFI: SpheryStream

EPSRC-SFI：SpheryStream

基本信息

批准号：
EP/T03324X/1
负责人：
Laura TONI
金额：
$ 32.81万
依托单位：
University College London
依托单位国家：
英国
项目类别：
Research Grant
财政年份：
2021
资助国家：
英国
起止时间：
2021 至无数据
项目状态：
已结题

来源：
https://gtr.ukri.org/projects?ref=EP%2FT03324X%2F1
关键词：
EPSRC SFI SpheryStream

项目摘要

This proposal addresses the important and not well-studied area in the fast-developing engineering space of the virtual reality and immersive technology. Our final vision is to endow any final user with an unprecedented sense of full immersion in virtual environments for any applicative scenario and with any connectivity level. This requires VR systems that operate at scale, in a personalized manner, remaining bandwidth-tolerant whilst meeting quality and latency criteria. This can be accomplished only by a fundamental revolution of the coding/streaming/rendering chain that has to put the interactive user at the heart of the system rather than at the end of the chain.Specifically, bringing our final vision to reality requires constant innovation in VR streaming technology to overcome the following main challenges: i) the spherical format of the video content in a scenario in which media codecs are optimized for planar content; ii) the interactive behaviour of users that introduces uncertainty on the popularity of the video content. To overcome the former challenges, the spherical video is currently projected on a bidimensional planar domain, but this introduces noticeable deformation since the sphere is not a developable surface. To address the uncertainty of users' behaviours, predicting models have been investigated recently, but none of them actually work on the spherical (not deformed) domain. To overcome the above challenges in immersive technology, there is the need to develop an efficient tool for navigation patterns analysis in the spherical domain and leverage on that to predict users' behaviour and build the entire coding-delivery-rendering pipeline as a user- and geometry-centric system. This proposal focuses on develop novel VR delivery strategies, which will rely on the analysis and prediction of the navigation patterns of the omnidirectional content navigation, in a user, content- and application-dependent fashion. Predicting users' behaviour will allow to delivery only the content that will be actually consumed by the users, minimising the transmission resources consumption while maximising the user's Quality of Experience. The key novelty will be to address the above challenges working directly on the spherical domain, rather than on the projected (thus deformed) bidimensional domain. The developed VR system will be tested via a real omnidirectional video streaming testbed that will be implemented within the scope of the project. The outcomes of the project will be three-folds: i) user- content- and application-dependent VR streaming platform, optimized from the source codec to the delivery platform; ii) users analysis tools able to assess users' behaviour (when interactive with the content) in an objective fashion to identify common navigation patterns, and to predict future behaviours; iii) VR streaming testbed to test the proposed technology.

该提案探讨了虚拟现实和沉浸式技术快速发展的工程空间中重要且不良好的领域。我们的最终愿景是赋予任何最终用户对任何应用方案和任何连接级别的虚拟环境中充分沉浸感的前所未有的感觉。这就需要以个性化的方式按大规模运行的VR系统，并保持带宽耐受性，同时满足质量和延迟标准。这只能通过编码/流式/渲染链的基本革命才能实现，该革命必须使交互式用户处于系统的核心，而不是链接结束时。具体说明，使我们的最终愿景变为现实，需要在VR流媒体技术中持续创新，以克服以下主要挑战：i）i）i）i）i）在场景中的视频内容构成了媒体介绍媒体的领域。 ii）用户的互动行为引入了视频内容的流行不确定性。为了克服前者的挑战，球形视频目前投影在双度平面域上，但是由于球体不是可开发的表面，因此这引入了明显的变形。为了解决用户行为的不确定性，最近已经研究了预测模型，但是它们实际上都在球形（未变形）域上起作用。为了克服沉浸式技术的上述挑战，需要开发一个有效的工具，用于在球形领域中进行导航模式分析，并利用它来预测用户的行为并构建整个以用户和几何形式的系统为中心的编码递送供应管道。该建议着重于制定新颖的VR交付策略，该策略将依靠用户，内容和应用程序依赖性方式的全向内容导航的导航模式的分析和预测。预测用户的行为将允许用户实际消耗的内容交付，从而最大程度地减少传输资源的消耗，同时最大化用户的体验质量。关键的新颖性是应对直接在球形域上工作的上述挑战，而不是在投影（因此变形）二维领域上。开发的VR系统将通过实际的全向视频流测试床进行测试，该测试将在项目范围内实现。该项目的结果将为三倍：i）用户内容和应用程序依赖性VR流媒体平台，从源编解码器到交付平台进行了优化； ii）用户分析工具能够以客观的方式评估用户的行为（与内容交互时），以识别常见的导航模式并预测未来的行为； iii）VR流媒体测试台以测试所提出的技术。

项目成果

期刊论文数量（7）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Correlation between Entropy and Prediction Error in VR Head Motion Trajectories

DOI：
10.1145/3607546.3616805
发表时间：
2023-10
期刊：
Proceedings of the 2nd International Workshop on Interactive eXtended Reality
影响因子：
0
作者：
Silvia Rossi;Laura Toni;Pablo César
通讯作者：
Silvia Rossi;Laura Toni;Pablo César

Immersive Video Technologies

沉浸式视频技术