Collaborative Research: CNS Core: Medium: Foundations and Scalable Algorithms for Personalized and Collaborative Virtual Reality Over Wireless Networks

无线网络上的虚拟现实（VR）可以同时为多个用户提供交互式和沉浸式体验，因此具有许多应用，特别是在基于VR的教育/培训中。然而，这种无线沉浸式服务中令人满意的个性化用户体验需要严格的性能要求，包括：（1）高速和高分辨率的全景图像渲染;（2）极低的延迟保证;以及（3）无缝的用户体验。除了上述要求外，协同用户体验还需要VR服务的可扩展性和公平性。现有的VR系统严重依赖于各种启发式设计，并且没有有效地利用VR内容的共性及其可预测性，这阻碍了它们的大规模部署。该项目旨在开发一个通过无线网络提供个性化和可扩展的协作VR体验的系统的理论基础和完整实现。该项目将集成机器学习、无线网络和移动的计算，以在商用移动的设备上实现高质量和可扩展的无线沉浸式应用。本项目的理论和实际应用将被整合到本科和研究生课程中，并将让K-12学生接触到最先进的无线和VR技术。（2）用于多个用户的VR内容之间的通用性，以实现多播;以及（3）VR内容的可预测性，以实现预取。拟议的研究将有助于和推进无线网络和虚拟现实领域的理论和面向系统的研究。该项目明确利用沉浸式VR应用和无线网络的独特特性，并提出以下四个相互依赖的研究方向：（I）处理网络和预测不确定性：该方向将研究算法设计，以优化个性化用户体验，同时考虑网络和视口预测的不确定性。(II)满足严格的沉浸式服务要求：这一目标将开发无线调度算法，为多个VR用户提供严格的沉浸式个性化服务保证。(III)支持流畅的协作交互：这一重点将集中在算法设计上，该设计利用了协作交互过程中自然出现的VR内容相似性和可预测性。(IV)可扩展的系统集成、实施、评估和部署：该项目将把研究重点I至III整合到一个整体系统中，进行系统级优化，并通过实验室实验和现实世界的课堂部署进行评估。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

Virtual reality (VR) over wireless networks can provide an interactive and immersive experience for multiple users simultaneously and thus has many applications, especially in VR-based education/training. However, satisfactory personalized user experience in such wireless immersive services demands stringent performance requirements, including: (1) high-speed and high-resolution panoramic image rendering; (2) extremely low delay guarantees; and (3) seamless user experience. Besides the aforementioned requirements, collaborative user experience requires both scalability and fairness of VR service. Existing VR systems heavily rely on various heuristic designs and do not efficiently exploit VR content commonality and its predictability, which impede their large-scale deployment. This project aims to develop the theoretical foundations and complete implementation of a system for providing both personalized and scalable collaborative VR experience over wireless networks. This project will integrate machine learning, wireless networking, and mobile computing to enable high-quality and scalable wireless immersive applications on commodity mobile devices. The theory and practical implementations to be developed in this project will be integrated into both undergraduate and graduate curriculum, as well as exposing K-12 students to state-of-the-art wireless and VR technologies.The proposed designs are motivated by a number of insights that we have developed from our preliminary work, including (1) viewport-adaptive rendering; (2) commonality among VR content for multiple users to enable multicasting; and (3) predictability of VR content to enable prefetching. The proposed research will contribute to and advance both theoretical and system-oriented research in the fields of wireless networks and virtual reality. The project explicitly exploits the unique characteristics of both immersive VR applications and wireless networks, and propose the following four interdependent research thrusts: (I) Dealing with network and prediction uncertainties: This thrust will investigate algorithm designs to optimize personalized user experience given both network and viewport prediction uncertainties. (II) Meeting stringent immersive service requirements: This thrust will develop wireless scheduling algorithms that provide stringent immersive, personalized service guarantees for multiple VR users. (III) Supporting smooth collaborative interaction: This thrust will focus on the algorithm design that leverages the VR content similarities and predictabilities that naturally emerge during collaborative interactions. (IV) Scalable system integration, implementation, evaluation, and deployment: This thrust will integrate Research Thrusts I through III into a holistic system, perform system-level optimizations, and evaluate it through lab experiments and real-world classroom deployment.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.