SHF: Medium: A Visual Cloud for Virtual Reality Applications

通过在教育、旅游、远程呈现等领域实现强大的虚拟现实视频应用，收集视频的能力可以彻底改变与世界的互动。这些新应用涉及处理和提供360度立体视频，这需要技术上的巨大改进来管理和处理真正沉浸式体验所必需的大规模视觉数据。支持VR视频的系统也是一种优秀的教育工具，因为学生可以以真正身临其境的方式体验场景，从而更好地传达内容。该项目构建了一个视觉云，提供对新的数据库管理系统的无缝访问，该系统具有硬件加速和边缘计算，能够高效实时地管理大规模图像数据和构建在其上的虚拟现实（VR）应用程序。该项目为VR数据处理开发了一个新的硬件和软件堆栈，并在公共云中执行。该堆栈包括一个新的存储管理器，与现有系统相比，它显着提高了多维数组数据的数据摄取和检索吞吐量，这是由VR应用程序的极端需求驱动的。存储管理器利用新颖的硬件技术（非易失性存储器），并提供新颖的近似和多分辨率数据存储功能。该项目还开发了一个新的运行时系统，用于高吞吐量和大规模阵列处理，开发了一个新的API，用于将VR管道表示为用户定义函数图，一个针对不同类型硬件（CPU， GPU， FPGA和3D XPoint）的已知VR算法的专门实现库，以及相关的优化器和调度器。最后，该项目开发新技术以实现实时VR应用。它们包括用于实时VR视频处理的基于fpga的加速平台，以及用于在观看者附近预取和缓存VR数据并在观看设备中处理该数据的算法和软件组件。http://visualcloud.cs.washington.edu

The ability to collect videos can revolutionize interactions with the world by enabling powerful virtual reality video applications in education, tourism, tele-presence and others. These new applications involve processing and serving 360-degree stereoscopic videos, which require a dramatic improvement in technology to manage and process the massive-scale visual data necessary for truly immersive experiences. Systems that support VR video also represent an excellent educational tool as students can experience scenes in a truly immersive way and hence better convey content. This project builds a visual cloud that provides seamless access to a new database management system with hardware acceleration and edge computation that enables the efficient and real-time management of massive-scale image data and virtual reality (VR) applications built on top of it. The project develops a new hardware and software stack for VR data processing with execution in public clouds. The stack includes a new storage manager that significantly increases data ingest and retrieval throughputs for multidimensional array data compared with existing systems, as motivated by the extreme needs of VR applications. The storage manager utilizes novel hardware technologies (non-volatile memory) and provides novel approximate and multi-resolution data storage capabilities. The project also develops a new runtime system for high-throughput and large-scale array processing by developing a new API for expressing VR pipelines as a graph of user-defined functions, a library of specialized implementations of known VR algorithms for different types of hardware (CPU, GPU, FPGA, and 3D XPoint), and associated optimizers and schedulers. Finally, the project develops new techniques to enable real-time VR applications. They include an FPGA-based acceleration platform for real-time VR video processing and algorithms and software components for prefetching and caching VR data close to the viewers and processing that data in the viewing device. http://visualcloud.cs.washington.edu