Live 360 Degree Video Delivery Based on User Collaboration in a Streaming Flock
Live 360 Degree Video Delivery Based on User Collaboration in a Streaming Flock
复制标题
DOI：
10.1109/tmm.2022.3149642
复制
发表时间：
2023
影响因子：
7.3
通讯作者：
Liyang Sun;Yixiang Mao;Tongyu Zong;Yong Liu;Yao Wang
中科院分区：
文献类型：
--
作者：
Liyang Sun;Yixiang Mao;Tongyu Zong;Yong Liu;Yao Wang
360度直播视频的流媒体允许用户从任何角度关注直播事件，并且已经在一些商业平台上部署。然而，目前的系统只能以相对较低的质量传输视频，因为整个360度视频是在有限的带宽下传输给用户的。视频流落入用户视场可以提高360度视频传输的带宽效率。在本文中，我们提出使用“蜂拥”的思想来同时提高360度实时视频流的用户视场预测精度和视频传输效率。通过在流会话中根据用户的网络条件为其分配可变的播放延迟，形成一个“流群”，并由播放延迟低的“强”用户在群的前面引导。我们提出了一种基于长短期记忆（LSTM）的协同视场预测方案，该方案利用群前用户的视场轨迹来预测群后用户的视场。给定一个预测的视场，我们开发了一个最佳的速率分配策略，以最大限度地提高感知质量。通过使用真实用户FoV跟踪和LTE/ 5g网络带宽跟踪进行实验，我们在几个基准测试中评估了所提出策略的增益。实验结果表明，与启发式FoV预测策略相比，所提出的流媒体系统可以显著提高整体质量约10 dB。此外，网络感知的群集形成可以在不影响视频质量的情况下进一步减少视频冻结。
Streaming of live 360-degree video allows users to follow a live event from any view point and has already been deployed on some commercial platforms. However, the current systems can only stream the video at relatively low-quality because the entire 360-degree video is delivered to the users under limited bandwidth. Streaming video falling into user field of view (FoV) can improve bandwidth efficiency of 360-degree video delivery. In this paper, we propose to use the idea of “flocking” to simultaneously improve the accuracy of user FoV prediction and video delivery efficiency for live 360-degree video streaming. By assigning variable playback latencies to users in a streaming session based on their network conditions, a “streaming flock” is formed and led by “strong” users with low playback latencies in the front of the flock. We propose a long short-term memory (LSTM) based collaborative FoV prediction scheme where the FoV traces of users in the front of the flock are utilized to predict the FoV of users behind them. Given a predicted FoV, we develop an optimal rate allocation strategy to maximize the perceptual quality. By conducting experiments using real-world user FoV traces and LTE/5 G network bandwidth traces, we evaluate the gains of the proposed strategies over several benchmarks. Our experimental results demonstrate that the proposed streaming system can increase the overall quality dramatically by about 10 dB compared with heuristic FoV prediction strategy. In addition, the network-aware flocking formation can further reduce the video freeze without influencing video quality.