Cloud compression for live volumetric video

Consense Reality开发了一种系统，用于捕获实时体积视频并将其传输到游戏引擎中。这将允许客户在虚拟空间举办有门票的活动，这可以极大地扩大现场音乐的潜在市场。到目前为止，浓缩现实的焦点一直是现场捕捉。为了达到这个阶段，它已经解决了许多复杂的计算机视觉问题。这一点已在2021年12月345万GB的风险投资中得到验证，该公司现在正将重点转向分销和向最终用户交付身临其境的体验。为了做到这一点，需要保持高质量但降低从基于云的CDN流传输的实况体积视频的比特率。为了继续推动采用，需要满足具有多种连接速度和不同处理能力的设备的客户的需求。本项目致力于深度学习在实时视频压缩中的新应用。这是一个很难解决的问题。布里斯托尔大学是公认的基于人工智能的视频编码的世界领先者，对MPEG标准做出了贡献，包括新的训练数据库、编码框架和深度学习网络结构。在这个合作项目中，视觉信息实验室的研究人员将提供专业知识，并加强他们对体积视频内容的最先进的编码方法。**本项目将重点放在：**1.利用深度学习来提高压缩效率的新方法。用于以可变比特率提供体积视频内容的增强型架构。将系统集成到解码器中，根据带宽和设备能力动态影响码率(更新Unity和UnReal插件，使其能够应对任何压缩方法)。通过这一点，我们将为用户提供以下**好处**：1.增强的视觉质量和与体积视频内容相关的沉浸式体验2。减少了压缩系统的端到端延迟3。改进了交付的单位经济性4。增加内容可访问性该项目将对体积视频交付的最新水平做出重大贡献，并对各种身临其境的视频应用产生重大影响。布里斯托尔拥有一个由研究人员、技术公司和艺术家组成的独特生态系统，可以走在定义和创建未来协作性Metverse的前沿。该项目的世界领先创新将为这一雄心壮志做出贡献，从而为该地区带来显著的经济效益。

Condense Reality has developed a system for capturing and streaming live volumetric video into Game Engines. This will allow customers to put on ticketed events in virtual spaces that can hugely expand the addressable market for live music.To date the focus of Condense Reality has been live capture. It has solved many complex computer vision problems in order to get to this stage. This has been validated by a £3.45M VC investment in December 2021\.It is now shifting focus to distribution and the delivery of immersive experiences to the end user. In order to do this there is a need to maintain high quality but reduce the bitrate of the live volumetric video that is streamed from a cloud based CDN. In order to continue to drive adoption there is a need to cater for customers with a wide range of connection speeds and on devices with varying processing capabilities. This project is focused on the novel application of deep learning to efficiently compress live-streamed volumetric video. This is a hard problem to solve.University of Bristol is recognised as a world leader in AI-based video coding, with contributions to MPEG standards including new training databases, coding frameworks, and deep learning network architectures. In this collaborative project, researchers within the Visual Information Laboratory will contribute expertise and enhance their state-of-the-art coding approaches to volumetric video content.**This project will focus on:**1. Novel methods that employ deep learning to improve compression efficiency.2. Enhanced architectures for delivering volumetric video content at variable bitrates.3. Integrating systems into the decoder which dynamically influence the bitrate based on bandwidth and device capability (updating the Unity and Unreal plugins so that they can cope with whatever compression methodology is adopted).Through this, we will provide the following **benefits to the user**:1. Enhanced visual quality and immersive experience associated with volumetric video content2. Reduced end-to-end latency of the compression systems3. Improved unit economics of delivery4. Increased content accessibilityThis project will make a major contribution to the state of the art in volumetric video delivery and generate significant impact on a wide range of immersive video applications. Bristol has a unique ecosystem of researchers, technology companies and artists that can be at the forefront of defining and creating a future collaborative Metaverse. The world-leading innovations in this project will contribute to this ambition and thus bring with significant economic benefits to the region.