Collaborative Research: CNS Core: Small: From Capture to Consumption: System Challenges in Pervasive 360-Degree Video Sharing

点击翻译按钮获取中文摘要

360-degree video allows users to explore a recorded scene at any angle from the camera position. This greater flexibility provided by the 360-degree video format has led to its increased popularity. However, widespread adoption has slowed because of the large amounts of resources required by the format. This work proposes methods to improve efficiency at each stage of the 360-degree video sharing path, from video capture at the sender side to consumption at the receiver side. The project addresses these challenges via four research thrusts. The first thrust explores approaches for generating high-quality frames during real-time stitching of multiple videos captured using commodity lenses. The proposed approach applies gradient decent to refine an initial rough stitching. The second thrust investigates mechanisms to spatially adapt content to be uploaded based on the available bandwidth. The third thrust aims to optimize the fine-grained representation of omnidirectional content to achieve high projection efficiency and view efficiency. The last thrust uses edge computing techniques to optimize the look-ahead window size on the receiver side to allow bandwidth-efficient content downloading.The techniques developed during the course of the project will allow users to enjoy more efficient 360-degree video streaming applications. These techniques are not only applicable to 360-degree video streaming but will also have applications in future generations of virtual reality (VR) technologies. The project will also motivate general student interest in computer science research, directly train students during the course of the project’s research, and contribute to curriculum development. The project will produce publications, code, data, and other research artifacts. All such artifacts will be made available publicly through the URLs http://www.cs.gmu.edu/~sqchen/ and http://www.cs.binghamton.edu/~yaoliu/ during the course of the project and remain available for at least five years after completion of the project. The code will be open sourced and will also be made available at code hosting website, e.g., GitHub.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.