YuZu: Neural-Enhanced Volumetric Video Streaming
YuZu: Neural-Enhanced Volumetric Video Streaming
复制标题
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Anlan Zhang;Chendong Wang;B. Han;Feng Qian
中科院分区：
文献类型：
--
作者：
Anlan Zhang;Chendong Wang;B. Han;Feng Qian
与传统的2D视频区分，体积视频提供了真正的3D沉浸式观看体验，并使观众能够锻炼六个自由度（6DOF） ，我们建议利用3D超级分辨率（SR）大幅度提高体积视频流的视觉质量，以实现这一目标，我们对现成的3D SR模型进行了深度内部的框架间优化×SR推断没有准确性降解的速度。然后，我们将上述组件与重要功能（例如QOE驱动的网络/计算资源适应）一起整合到一个名为Yuzu的整体系统中，该系统执行线路速率（在30+ fps）进行自适应SR，以表明我们的评估表明我们的评估表明。与NO SR相比，Yuzu可以将体积视频流的QoE提高37％至178％，并且在QoE上的表现优于现有的视口自适应解决方案，至175％。
Differing from traditional 2D videos, volumetric videos provide true 3D immersive viewing experiences and allow viewers to exercise six degree-of-freedom (6DoF) motion. However, streaming high-quality volumetric videos over the Internet is extremely bandwidth-consuming. In this paper, we propose to leverage 3D super resolution (SR) to drastically increase the visual quality of volumetric video streaming. To accomplish this goal, we conduct deep intraand inter-frame optimizations for off-the-shelf 3D SR models, and achieve up to 542× speedup on SR inference without accuracy degradation. We also derive a first Quality of Experience (QoE) model for SR-enhanced volumetric video streaming, and validate it through extensive user studies involving 1,446 subjects, achieving a median QoE estimation error of 12.49%. We then integrate the above components, together with important features such as QoE-driven network/compute resource adaptation, into a holistic system called YuZu that performs line-rate (at 30+ FPS) adaptive SR for volumetric video streaming. Our evaluations show that YuZu can boost the QoE of volumetric video streaming by 37% to 178% compared to no SR, and outperform existing viewport-adaptive solutions by 101% to 175% on QoE.