Unlocking 360° Virtual Reality (VR) by Spherical Deep Learning

今天的虚拟现实（VR）体验要么提供照片真实感，要么提供交互性，但不能同时提供这两种体验。基于360°视频内容的VR体验是真实的，因为它们是基于摄影的。然而，这些体验并不具有互动性，因为它们是基于给定位置的摄像机镜头，因此用户无法在虚拟场景中移动。另一种基于计算机生成图像（CGI）的虚拟现实体验确实提供了交互性，但通常远非照片真实感。在这个项目中，Kagenova正在开发核心技术，以同时大规模地提供照片真实感和交互性。为了达到更高的真实感和沉浸感，我们正在开发新的球面数据深度学习技术，例如360°照片和视频。我们的技术允许用户进入360°VR视频并在场景中移动。我们将360°视频内容转变为可探索的虚拟世界，从而使VR内容创作民主化，这样任何人都可以用廉价的消费级360°相机创建逼真的沉浸式内容。

Today's virtual reality (VR) experiences provide either photo-realism or interactivity --- but not both at the same time. VR experiences based on 360° video content are photo-realistic since they are based on photography. However, these experiences are not interactive since they are based on footage from a camera at a given position and hence the user cannot move about in the virtual scene. Alternative VR experiences based on computer generated imagery (CGI) do provide interactivity but are usually far from photo-realistic. In this project Kagenova is developing core technology to provide photo-realism and interactivity at the same time and at scale. To achieve this elevated level of realism and immersion, we are developing novel deep learning techniques for spherical data, such as 360° photos and videos. Our technology allows users to walk into 360° VR video and move about in the scene. We turn 360° video content into explorable virtual worlds, thereby democratising VR content creation so that life-like immersive content can be created by anyone with a cheap, consumer-grade 360° camera.