Toward Next-generation Volumetric Video Streaming with Neural-based Content Representations
Toward Next-generation Volumetric Video Streaming with Neural-based Content Representations
复制标题
DOI：
10.1145/3615452.3617938
复制
发表时间：
2023-10
期刊：
影响因子：
--
通讯作者：
中科院分区：
文献类型：
--
作者：
在最小化带宽消耗和保持高视觉质量之间取得平衡是批量内容交付的首要目标。然而，实现这一雄心勃勃的目标是一个巨大的挑战，特别是对于具有有限的计算资源的移动的设备，考虑到要流传输的大量3D数据、严格的延迟要求和高计算负载。受神经辐射场（NeRF）提供的优势的启发，我们首次提出通过利用基于神经的内容表示来提供体积视频。我们深入研究了潜在的挑战，并为视频点播（VOD）和直播视频流服务探索了可行的解决方案，包括端到端管道、实时和高质量流、速率自适应和视口自适应。我们的初步结果使我们的研究命题的可行性，提供了一个有前途的起点，为进一步的调查。
Striking a balance between minimizing bandwidth consumption and maintaining high visual quality stands as the paramount objective in volumetric content delivery. However, achieving this ambitious target is a substantial challenge, especially for mobile devices with constrained computational resources, given the voluminous amount of 3D data to be streamed, strict latency requirements, and high computational load. Inspired by the advantages offered by neural radiance fields (NeRF), we propose, for the first time, to deliver volumetric videos by utilizing neural-based content representations. We delve deep into potential challenges and explore viable solutions for both video-on-demand (VOD) and live video streaming services, in terms of the end-to-end pipeline, real-time and high-quality streaming, rate adaptation, and viewport adaptation. Our preliminary results lend credence to the feasibility of our research proposition, offering a promising starting point for further investigation.