In-Network View Synthesis for Interactive Multiview Video Systems
In-Network View Synthesis for Interactive Multiview Video Systems
复制标题
DOI：
10.1109/tmm.2016.2537207
复制
发表时间：
2015-09
影响因子：
7.3
通讯作者：
L. Toni;Gene Cheung;P. Frossard
中科院分区：
文献类型：
--
作者：
L. Toni;Gene Cheung;P. Frossard
在多视图应用中，摄像机视图可用作参考视图来合成其他虚拟视点，从而允许用户在 3D 场景中自由导航。然而，带宽限制可能会限制发送给客户端的参考视图的数量，从而限制合成视点的质量。在这项工作中，我们研究了网络内参考视图合成问题，旨在提高客户端的导航质量。我们考虑一种分布式云网络架构，其中存储在主云中的数据在小云（即靠近用户的资源丰富的代理）的帮助下传递给最终用户。我们认为，在从 Cloudlet 到用户的带宽有限的情况下，与仅对原始摄像机视图集进行子采样相比，在 Cloudlet 处对 3D 场景的视点进行重新采样（即，在 Cloudlet 中合成新的虚拟视图以用作解码器的新参考）是有益的。因此，我们提出了一个新的参考视图选择问题，该问题寻求视图子集，在带宽限制下最小化用户定义的视图导航窗口上的失真。我们证明了该问题是NP-hard问题，并提出了一种使用动态规划的有效多项式时间算法来解决在涵盖实践中大多数多视图场景的一般假设下的优化问题。仿真结果证实了网络中虚拟视图合成所提供的性能增益。
In multiview applications, camera views can be used as reference views to synthesize additional virtual viewpoints, allowing users to freely navigate within a 3D scene. However, bandwidth constraints may restrict the number of reference views sent to clients, limiting the quality of the synthesized viewpoints. In this work, we study the problem of in-network reference view synthesis aimed at improving the navigation quality at the clients. We consider a distributed cloud network architecture, where data stored in a main cloud is delivered to end users with the help of cloudlets, i.e., resource-rich proxies close to the users. We argue that, in case of limited bandwidth from the cloudlet to the users, re-sampling at the couldlet the viewpoints of the 3D scene (i.e., synthesizing novel virtual views in the cloudlets to be used as new references to the decoder) is beneficial compared to mere subsampling of the original set of camera views. We therefore cast a new reference view selection problem that seeks the subset of views minimizing the distortion over a view navigation window defined by the user under bandwidth constraints. We prove that the problem is NP-hard, and we propose an effective polynomial time algorithm using dynamic programming to solve the optimization problem under general assumptions that cover most of the multiview scenarios in practice. Simulation results confirm the performance gain offered by virtual view synthesis in the network.