Collaborative Research: CNS Core: Medium: Foundations and Scalable Algorithms for Personalized and Collaborative Virtual Reality Over Wireless Networks

基于无线网络的虚拟现实(VR)可以同时为多个用户提供交互和身临其境的体验，因此在基于VR的教育/培训中有着广泛的应用。然而，在这种无线沉浸式服务中，令人满意的个性化用户体验需要严格的性能要求，包括：(1)高速和高分辨率全景图像渲染；(2)极低的延迟保证；(3)无缝用户体验。除了上述要求外，协同用户体验还要求VR服务的可扩展性和公平性。现有的VR系统严重依赖于各种启发式设计，没有有效地利用VR内容的通用性和可预测性，这阻碍了它们的大规模部署。该项目旨在开发一个在无线网络上提供个性化和可扩展的协作VR体验的系统的理论基础和完整的实现。该项目将集成机器学习、无线网络和移动计算，以在商用移动设备上实现高质量和可扩展的无线沉浸式应用程序。该项目将开发的理论和实践实施将被整合到本科和研究生课程中，并向K-12学生展示最先进的无线和VR技术。建议的设计是基于我们从初步工作中获得的一些见解，包括(1)视点自适应渲染；(2)多用户VR内容的共性以实现多播；以及(3)VR内容的可预测性以实现预取。所提出的研究将有助于和推动无线网络和虚拟现实领域的理论和面向系统的研究。该项目明确利用了沉浸式VR应用和无线网络的独特特性，并提出了以下四个相互依赖的研究主题：(I)处理网络和预测不确定性：此主题将研究在网络和视点预测不确定性的情况下优化个性化用户体验的算法设计。(Ii)满足严格的身临其境服务要求：这一推动力将开发无线调度算法，为多个VR用户提供严格的身临其境、个性化的服务保证。(Iii)支持顺畅的协作互动：这一重点将集中在算法设计上，该算法利用了协作互动过程中自然出现的VR内容的相似性和可预测性。(Iv)可扩展的系统集成、实施、评估和部署：这一努力将把研究推进I到III整合到一个整体系统中，执行系统级优化，并通过实验室实验和真实世界的课堂部署对其进行评估。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Virtual reality (VR) over wireless networks can provide an interactive and immersive experience for multiple users simultaneously and thus has many applications, especially in VR-based education/training. However, satisfactory personalized user experience in such wireless immersive services demands stringent performance requirements, including: (1) high-speed and high-resolution panoramic image rendering; (2) extremely low delay guarantees; and (3) seamless user experience. Besides the aforementioned requirements, collaborative user experience requires both scalability and fairness of VR service. Existing VR systems heavily rely on various heuristic designs and do not efficiently exploit VR content commonality and its predictability, which impede their large-scale deployment. This project aims to develop the theoretical foundations and complete implementation of a system for providing both personalized and scalable collaborative VR experience over wireless networks. This project will integrate machine learning, wireless networking, and mobile computing to enable high-quality and scalable wireless immersive applications on commodity mobile devices. The theory and practical implementations to be developed in this project will be integrated into both undergraduate and graduate curriculum, as well as exposing K-12 students to state-of-the-art wireless and VR technologies.The proposed designs are motivated by a number of insights that we have developed from our preliminary work, including (1) viewport-adaptive rendering; (2) commonality among VR content for multiple users to enable multicasting; and (3) predictability of VR content to enable prefetching. The proposed research will contribute to and advance both theoretical and system-oriented research in the fields of wireless networks and virtual reality. The project explicitly exploits the unique characteristics of both immersive VR applications and wireless networks, and propose the following four interdependent research thrusts: (I) Dealing with network and prediction uncertainties: This thrust will investigate algorithm designs to optimize personalized user experience given both network and viewport prediction uncertainties. (II) Meeting stringent immersive service requirements: This thrust will develop wireless scheduling algorithms that provide stringent immersive, personalized service guarantees for multiple VR users. (III) Supporting smooth collaborative interaction: This thrust will focus on the algorithm design that leverages the VR content similarities and predictabilities that naturally emerge during collaborative interactions. (IV) Scalable system integration, implementation, evaluation, and deployment: This thrust will integrate Research Thrusts I through III into a holistic system, perform system-level optimizations, and evaluate it through lab experiments and real-world classroom deployment.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.