Collaborative Research: CNS Core: Medium: Foundations and Scalable Algorithms for Personalized and Collaborative Virtual Reality Over Wireless Networks

无线网络上的虚拟现实（VR）可以同时为多个用户提供交互式和沉浸式体验，因此具有许多应用，特别是在基于VR的教育/培训中。然而，在这种无线沉浸式服务中，满足个性化的用户体验需要严格的性能要求，包括：(1)高速、高分辨率全景图像渲染；(2)极低的延迟保证；(3)无缝的用户体验。除了上述需求外，协同用户体验还需要VR服务的可扩展性和公平性。现有的虚拟现实系统严重依赖各种启发式设计，未能有效利用虚拟现实内容的通用性和可预测性，阻碍了虚拟现实系统的大规模部署。本项目旨在为通过无线网络提供个性化和可扩展的协同VR体验的系统提供理论基础和完整实现。该项目将整合机器学习、无线网络和移动计算，在商用移动设备上实现高质量和可扩展的无线沉浸式应用。该项目开发的理论和实践实施将整合到本科和研究生课程中，并使K-12学生接触到最先进的无线和VR技术。提出的设计是由我们从初步工作中开发的许多见解所驱动的，包括(1)视口自适应渲染；(2)多用户的VR内容通用性，实现多播；(3) VR内容的可预测性，以实现预取。所提出的研究将有助于和推进无线网络和虚拟现实领域的理论和系统导向研究。该项目明确地利用了沉浸式VR应用和无线网络的独特特征，并提出了以下四个相互依存的研究重点：(I)处理网络和预测不确定性：该重点将研究在网络和视口预测不确定性的情况下优化个性化用户体验的算法设计。（二）满足严格的沉浸式服务要求：该推力将开发无线调度算法，为多个VR用户提供严格的沉浸式个性化服务保障。（三）支持流畅的协同交互：这一重点将集中在算法设计上，利用协同交互过程中自然出现的VR内容相似性和可预测性。（四）可扩展的系统集成、实施、评估和部署：该推力将把研究推力I到III整合成一个整体系统，进行系统级优化，并通过实验室实验和现实世界的课堂部署对其进行评估。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Virtual reality (VR) over wireless networks can provide an interactive and immersive experience for multiple users simultaneously and thus has many applications, especially in VR-based education/training. However, satisfactory personalized user experience in such wireless immersive services demands stringent performance requirements, including: (1) high-speed and high-resolution panoramic image rendering; (2) extremely low delay guarantees; and (3) seamless user experience. Besides the aforementioned requirements, collaborative user experience requires both scalability and fairness of VR service. Existing VR systems heavily rely on various heuristic designs and do not efficiently exploit VR content commonality and its predictability, which impede their large-scale deployment. This project aims to develop the theoretical foundations and complete implementation of a system for providing both personalized and scalable collaborative VR experience over wireless networks. This project will integrate machine learning, wireless networking, and mobile computing to enable high-quality and scalable wireless immersive applications on commodity mobile devices. The theory and practical implementations to be developed in this project will be integrated into both undergraduate and graduate curriculum, as well as exposing K-12 students to state-of-the-art wireless and VR technologies.The proposed designs are motivated by a number of insights that we have developed from our preliminary work, including (1) viewport-adaptive rendering; (2) commonality among VR content for multiple users to enable multicasting; and (3) predictability of VR content to enable prefetching. The proposed research will contribute to and advance both theoretical and system-oriented research in the fields of wireless networks and virtual reality. The project explicitly exploits the unique characteristics of both immersive VR applications and wireless networks, and propose the following four interdependent research thrusts: (I) Dealing with network and prediction uncertainties: This thrust will investigate algorithm designs to optimize personalized user experience given both network and viewport prediction uncertainties. (II) Meeting stringent immersive service requirements: This thrust will develop wireless scheduling algorithms that provide stringent immersive, personalized service guarantees for multiple VR users. (III) Supporting smooth collaborative interaction: This thrust will focus on the algorithm design that leverages the VR content similarities and predictabilities that naturally emerge during collaborative interactions. (IV) Scalable system integration, implementation, evaluation, and deployment: This thrust will integrate Research Thrusts I through III into a holistic system, perform system-level optimizations, and evaluate it through lab experiments and real-world classroom deployment.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.