Semantic-Aware View Prediction for 360-Degree Videos at the 5G Edge
Semantic-Aware View Prediction for 360-Degree Videos at the 5G Edge
复制标题
DOI：
10.1109/ism55400.2022.00025
复制
发表时间：
2022-12
期刊：
影响因子：
--
通讯作者：
Shivi Vats;Jounsup Park;K. Nahrstedt;M. Zink;R. Sitaraman;H. Hellwagner
中科院分区：
文献类型：
--
作者：
Shivi Vats;Jounsup Park;K. Nahrstedt;M. Zink;R. Sitaraman;H. Hellwagner
在 5G 测试台中，我们使用 360° 视频流来测试、测量和演示 5G 基础设施，包括边缘计算支持的功能和挑战。具体来说，我们使用最初在[1]中描述的SEAWARE（语义感知视图预测）软件系统，在5G网络边缘通过视图预测来支持360°视频播放器（处理平铺视频）。最初，SEAWARE 通过提取重要对象和事件等方式对媒体服务器上的 360° 视频进行语义分析。该视频语义信息以特定的数据结构进行编码，并在 DASH 流媒体框架中与客户端共享。利用这些数据结构，客户端/播放器可以执行视图预测，而无需进行深入的、计算成本高昂的语义视频分析。在本文中，SEAWARE 系统被移植并适应于（部分）在边缘运行，它可用于预测视图并以高质量预取预测的片段/图块，以便在需要时让它们在靠近客户端的地方可用。本文概述了 5G 测试床、整体架构以及 SEAWARE 在边缘服务器的实现。由于这项工作的一个重要目标是实现低运动到玻璃的延迟，因此我们开发并描述了“图块后加载”，这是一种允许将非预测图块以高质量提取到播放器缓冲区中已有的片段中的技术。评估并展示了 5G 基础设施上 360° 平铺视频播放的性能。指出了当前使用的 5G 网络的局限性以及基于 DASH 的流媒体和边缘辅助视口预测在“现实世界”约束下的一些挑战；此外，还公开了切片后加载的性能优势。
In a 5G testbed, we use 360° video streaming to test, measure, and demonstrate the 5G infrastructure, including the capabilities and challenges of edge computing support. Specifically, we use the SEAWARE (Semantic-Aware View Prediction) software system, originally described in [1], at the edge of the 5G network to support a 360° video player (handling tiled videos) by view prediction. Originally, SEAWARE performs semantic analysis of a 360° video on the media server, by extracting, e.g., important objects and events. This video semantic information is encoded in specific data structures and shared with the client in a DASH streaming framework. Making use of these data structures, the client/player can perform view prediction without in-depth, computationally expensive semantic video analysis. In this paper, the SEAWARE system was ported and adapted to run (partially) on the edge where it can be used to predict views and prefetch predicted segments/tiles in high quality in order to have them available close to the client when requested. The paper gives an overview of the 5G testbed, the overall architecture, and the implementation of SEAWARE at the edge server. Since an important goal of this work is to achieve low motion-to-glass latencies, we developed and describe "tile postloading", a technique that allows non-predicted tiles to be fetched in high quality into a segment already available in the player buffer. The performance of 360° tiled video playback on the 5G infrastructure is evaluated and presented. Current limitations of the 5G network in use and some challenges of DASH-based streaming and of edge-assisted viewport prediction under "real-world" constraints are pointed out; further, the performance benefits of tile postloading are disclosed.