Viewing the 360° Future: Trade-Off Between User Field-of-View Prediction, Network Bandwidth, and Delay
Viewing the 360° Future: Trade-Off Between User Field-of-View Prediction, Network Bandwidth, and Delay
复制标题
DOI：
10.1109/icccn49398.2020.9209659
复制
发表时间：
2020-08
期刊：
影响因子：
--
通讯作者：
Shahryar Afzal;Jiasi Chen;K. K. Ramakrishnan-K.
中科院分区：
文献类型：
--
作者：
Shahryar Afzal;Jiasi Chen;K. K. Ramakrishnan-K.
准确预测用户的视场（FoV）可以帮助显著降低360°视频流的高带宽要求，因为它使得能够仅发送与预测的FoV相对应的瓦片。由于用于用户头部定向的许多方法（即，FoV）预测，从简单的线性回归到更复杂的神经网络，很难全面地决定使用哪种方法。为了解决这一知识差距，在这项工作中，我们基准用户预测算法在多个数据集的聚合，并研究这种分析的影响。我们的研究结果表明，它确实是很难的任何预测算法，以准确地预测用户的FoV超过一个非常短的未来时间窗口约300毫秒。我们还观察到，用户的观看行为是由侧向头部运动为主，而不是上下。这些发现对客户端处的网络带宽、延迟和回放缓冲具有影响：（1）在用户的FoV周围需要额外的“填充”区块以便校正预测误差;具体地，对于相同的带宽使用，矩形填充实现比方形填充更低的停滞率;（2）视频播放缓冲器、网络延迟和抖动需要很小，以避免对用户视场的陈旧预测，这些预测仅在未来300 ms内有效;（3）填充的每视频和每用户个性化可以为缓慢移动的用户或视频节省带宽。我们在数学上量化这些权衡，并提出模拟结果来证明这些发现和影响。我们的研究结果对未来360°流媒体系统中的FoV预测方法具有影响。
Predicting a user’s field-of-view (FoV) accurately can help to significantly reduce the high bandwidth requirements for 360° video streaming, as it enables sending only the tiles corresponding to the predicted FoV. Since many approaches for user head-orientation (i.e., FoV) prediction have been proposed in the literature, ranging from simple linear regression to more complex neural networks, it is difficult to comprehensively decide which method to use. Towards resolving this gap in knowledge, in this work we benchmark user prediction algorithms over an aggregation of multiple datasets and study the implications of this analysis. Our results demonstrate that it is indeed difficult for any prediction algorithm to accurately predict a user’s FoV beyond a very short future time window of approximately 300 ms. We also observe that users’ viewing behavior is dominated by sideways head movement, rather than up-and-down. These findings have implications on network bandwidth, latency, and playback buffering at the client: (1) Extra "padding" tiles are needed around the user’s FoV in order to correct for prediction errors; in particular, a rectangular padding achieves lower stall rate than square padding, for the same bandwidth usage; (2) Video playout buffers, network delay, and jitter need to be small in order to avoid stale predictions of the user’s field-of-view, which are only valid 300 ms into the future; (3) Per-video and per-user personalization of the padding can save bandwidth for slow-moving users or videos. We mathematically quantify these tradeoffs and present simulation results to demonstrate these findings and implications. Our results have implications for FoV prediction methods in future 360° streaming systems.