Sparse Directed Graph Learning for Head Movement Prediction in 360 Video Streaming
Sparse Directed Graph Learning for Head Movement Prediction in 360 Video Streaming
复制标题
DOI：
10.1109/icassp40776.2020.9053598
复制
发表时间：
2020-05
期刊：
影响因子：
--
通讯作者：
Xue Zhang;Gene Cheung;P. Callet;Jack Z. G. Tan
中科院分区：
文献类型：
--
作者：
Xue Zhang;Gene Cheung;P. Callet;Jack Z. G. Tan
以高质量编码的高清360度视频通常尺寸太大，无法在带宽（BW）受限的网络上完整地传输。一种流行的补救方法是交互式地提取并发送与头戴式显示器（HMD）中观看者当前视场（FoV）相对应的空间子区域，以获得更高的bw效率流。由于服务器和客户端之间不可忽略的往返时间（RTT）延迟，准确的头部运动预测预测观众未来的fov是必不可少的。现有的方法要么在建模方面过于简单化，并且在RTT较大时预测不佳，要么过度依赖数据驱动的学习，导致模型不灵活，对RTT异质性不具有鲁棒性。在本文中，我们将头部运动预测任务作为一个稀疏有向图学习问题，其中三个相关信息来源- 360图像显著性图，收集的观众头部运动轨迹和生物头部旋转模型-聚合到一个统一的马尔可夫模型中。具体来说，我们制定了一个约束优化问题，以最小化一个12范数保真度项和一个稀疏性项，分别对应于跟踪数据/显著性一致性和稀疏图模型先验。我们采用混合迭代加权最小二乘（IRLS）和Frank-Wolfe优化策略交替求解。大量的实验表明，我们的头部运动预测方案在广泛的rtt范围内明显优于现有的建议。
High-definition 360 videos encoded in fine quality are typically too large in size to stream in its entirety over bandwidth (BW)-constrained networks. One popular remedy is to interactively extract and send a spatial sub-region corresponding to a viewer’s current field-of-view (FoV) in a head-mounted display (HMD) for more BW-efficient streaming. Due to the non-negligible round-trip-time (RTT) delay between server and client, accurate head movement prediction that foretells a viewer’s future FoVs is essential. Existing approaches are either overly simplistic in modelling and predict poorly when RTT is large, or are over-reliant on data-driven learning, resulting in inflexible models that are not robust to RTT heterogeneity. In this paper, we cast the head movement prediction task as a sparse directed graph learning problem, where three sources of relevant information—a 360 image saliency map, collected viewers’ head movement traces, and a biological head rotation model—are aggregated into a unified Markov model. Specifically, we formulate a constrained optimization problem to minimize an l2-norm fidelity term and a sparsity term, corresponding to trace data / saliency consistency and a sparse graph model prior respectively. We solve the problem alternately using a hybrid iterative reweighted least square (IRLS) and Frank-Wolfe optimization strategy. Extensive experiments show that our head movement prediction scheme noticeably outperforms existing proposals across a wide range of RTTs.