LiveROI: region of interest analysis for viewport prediction in live mobile virtual reality streaming
LiveROI: region of interest analysis for viewport prediction in live mobile virtual reality streaming
复制标题
DOI：
10.1145/3458305.3463378
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
Xianglong Feng;Weitian Li;Sheng Wei
中科院分区：
文献类型：
--
作者：
Xianglong Feng;Weitian Li;Sheng Wei
虚拟现实(VR)流媒体可以为终端用户提供身临其境的视频观看体验，但会消耗巨大的带宽。最近的研究已经采用选择性流传输来解决带宽挑战，它以高质量预测和流传输用户感兴趣的视区，而以低质量预测和流传输视频的其他部分。然而，现有的视点预测机制主要针对视频点播(VOD)场景，依靠历史视频和用户跟踪数据构建预测模型。社区仍然缺乏有效的视点预测方法来支持VR直播，这是最吸引人和最受欢迎的VR流媒体体验。我们开发了一种基于感兴趣区域(ROI)的视点预测方法，即LiveROI，用于实时VR流媒体。LiveROI使用动作识别算法对视频内容进行分析，并将分析结果作为视点预测的基础。为了消除对历史视频/用户数据的需求，LiveROI采用自适应用户偏好建模和单词嵌入，在运行时根据用户头部方向动态选择视频视点。我们使用从公共VR头部运动数据集中获得的48个用户观看的12个VR视频来评估LiveROI。结果表明，LiveROI通过实时处理支持VR直播，实现了较高的预测精度和显著的带宽节约。
Virtual reality (VR) streaming can provide immersive video viewing experience to the end users but with huge bandwidth consumption. Recent research has adopted selective streaming to address the bandwidth challenge, which predicts and streams the user's viewport of interest with high quality and the other portions of the video with low quality. However, the existing viewport prediction mechanisms mainly target the video-on-demand (VOD) scenario relying on historical video and user trace data to build the prediction model. The community still lacks an effective viewport prediction approach to support live VR streaming, the most engaging and popular VR streaming experience. We develop a region of interest (ROI)-based viewport prediction approach, namely LiveROI, for live VR streaming. LiveROI employs an action recognition algorithm to analyze the video content and uses the analysis results as the basis of viewport prediction. To eliminate the need of historical video/user data, LiveROI employs adaptive user preference modeling and word embedding to dynamically select the video viewport at runtime based on the user head orientation. We evaluate LiveROI with 12 VR videos viewed by 48 users obtained from a public VR head movement dataset. The results show that LiveROI achieves high prediction accuracy and significant bandwidth savings with real-time processing to support live VR streaming.