High-resolution 360 Video Foveated Stitching for Real-time VR
High-resolution 360 Video Foveated Stitching for Real-time VR
复制标题
DOI：
10.1111/cgf.13277
复制
发表时间：
2017-10-01
影响因子：
2.5
通讯作者：
Chen, Bing-Yu
中科院分区：
文献类型：
--
作者：
Lee, Wei-Tse;Chen, Hsin-I;Chen, Bing-Yu
在虚拟现实（VR）应用中，通常通过创建真实世界场景的360度视频全景来生成内容。虽然许多捕获设备正在发布，但由于其计算要求高的性质，获得高分辨率的全景图和实时显示虚拟世界仍然具有挑战性。在本文中，我们提出了一个实时360度视频中心凹拼接框架，渲染整个场景在不同的细节层次，旨在创建一个高分辨率的全景视频实时，可以直接流到客户端。我们的中心凹拼接算法将来自多个摄像机的视频作为输入，结合人类视觉注意力的测量（即敏锐度图和显著性图），可以大大减少要处理的像素数量。我们使用GPU进一步并行化算法，以实现响应式界面，并通过用户研究验证我们的结果。我们的系统在Google Cardboard显示器上将图形计算加速了6倍。
In virtual reality (VR) applications, the contents are usually generated by creating a 360 degrees Video panorama of a real-world scene. Although many capture devices are being released, getting high-resolution panoramas and displaying a virtual world in real-time remains challenging due to its computationally demanding nature. In this paper, we propose a real-time 360 degrees Video foveated stitching framework, that renders the entire scene in different level of detail, aiming to create a high-resolution panoramic Video in real-time that can be streamed directly to the client. Our foveated stitching algorithm takes Videos from multiple cameras as input, combined with measurements of human visual attention (i.e. the acuity map and the saliency map), can greatly reduce the number of pixels to be processed. We further parallelize the algorithm using GPU to achieve a responsive interface and validate our results via a user study. Our system accelerates graphics computation by a factor of 6 on a Google Cardboard display.