PanoSynthVR: Toward Light-weight 360-Degree View Synthesis from a Single Panoramic Input
PanoSynthVR: Toward Light-weight 360-Degree View Synthesis from a Single Panoramic Input
复制标题
DOI：
10.1109/ismar55827.2022.00075
复制
发表时间：
2022-10
期刊：
影响因子：
--
通讯作者：
John Waidhofer;Richa Gadgil;Anthony Dickson;S. Zollmann;Jonathan Ventura
中科院分区：
文献类型：
--
作者：
John Waidhofer;Richa Gadgil;Anthony Dickson;S. Zollmann;Jonathan Ventura
我们研究了如何从单个全景图像输入在当前虚拟现实硬件上实现实时360°视图合成。我们引入了一个轻量级的方法来自动转换成一个多缸的图像表示，支持实时，自由视点视图合成渲染的虚拟现实的一个单一的全景输入。我们将现有的在针孔图像上训练的卷积神经网络应用于具有包裹填充的圆柱形全景图，以确保左右边缘之间的一致性。该网络输出一堆不同深度的半透明贴图，这些贴图可以很容易地渲染和合成。定量实验和用户研究表明，该方法产生令人信服的视差和较少的文物比纹理网格表示。
We investigate how real-time, 360° view synthesis can be achieved on current virtual reality hardware from a single panoramic image input. We introduce a light-weight method to automatically convert a single panoramic input into a multi-cylinder image representation that supports real-time, free-viewpoint view synthesis rendering for virtual reality. We apply an existing convolutional neural network trained on pinhole images to a cylindrical panorama with wrap padding to ensure agreement between the left and right edges. The network outputs a stack of semi-transparent panoramas at varying depths which can be easily rendered and composited with over blending. Quantitative experiments and a user study show that the method produces convincing parallax and fewer artifacts than a textured mesh representation.