Unlocking 360° Virtual Reality (VR) by Spherical Deep Learning

点击翻译按钮获取中文摘要

Today's virtual reality (VR) experiences provide either photo-realism or interactivity --- but not both at the same time. VR experiences based on 360° video content are photo-realistic since they are based on photography. However, these experiences are not interactive since they are based on footage from a camera at a given position and hence the user cannot move about in the virtual scene. Alternative VR experiences based on computer generated imagery (CGI) do provide interactivity but are usually far from photo-realistic. In this project Kagenova is developing core technology to provide photo-realism and interactivity at the same time and at scale. To achieve this elevated level of realism and immersion, we are developing novel deep learning techniques for spherical data, such as 360° photos and videos. Our technology allows users to walk into 360° VR video and move about in the scene. We turn 360° video content into explorable virtual worlds, thereby democratising VR content creation so that life-like immersive content can be created by anyone with a cheap, consumer-grade 360° camera.