Collaborative Research: CNS Core: Medium: Foundations and Scalable Algorithms for Personalized and Collaborative Virtual Reality Over Wireless Networks

协作研究：CNS 核心：中：无线网络上个性化和协作虚拟现实的基础和可扩展算法

• 批准号: 2106090
• 负责人: Feng Qian
• 金额: $ 26.6万
• 依托单位: University of Minnesota-Twin Cities
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2106090&HistoricalAwards=false
• 关键词: Collaborative; Research; CNS; Core; Medium

Virtual reality (VR) over wireless networks can provide an interactive and immersive experience for multiple users simultaneously and thus has many applications, especially in VR-based education/training. However, satisfactory personalized user experience in such wireless immersive services demands stringent performance requirements, including: (1) high-speed and high-resolution panoramic image rendering; (2) extremely low delay guarantees; and (3) seamless user experience. Besides the aforementioned requirements, collaborative user experience requires both scalability and fairness of VR service. Existing VR systems heavily rely on various heuristic designs and do not efficiently exploit VR content commonality and its predictability, which impede their large-scale deployment. This project aims to develop the theoretical foundations and complete implementation of a system for providing both personalized and scalable collaborative VR experience over wireless networks. This project will integrate machine learning, wireless networking, and mobile computing to enable high-quality and scalable wireless immersive applications on commodity mobile devices. The theory and practical implementations to be developed in this project will be integrated into both undergraduate and graduate curriculum, as well as exposing K-12 students to state-of-the-art wireless and VR technologies.The proposed designs are motivated by a number of insights that we have developed from our preliminary work, including (1) viewport-adaptive rendering; (2) commonality among VR content for multiple users to enable multicasting; and (3) predictability of VR content to enable prefetching. The proposed research will contribute to and advance both theoretical and system-oriented research in the fields of wireless networks and virtual reality. The project explicitly exploits the unique characteristics of both immersive VR applications and wireless networks, and propose the following four interdependent research thrusts: (I) Dealing with network and prediction uncertainties: This thrust will investigate algorithm designs to optimize personalized user experience given both network and viewport prediction uncertainties. (II) Meeting stringent immersive service requirements: This thrust will develop wireless scheduling algorithms that provide stringent immersive, personalized service guarantees for multiple VR users. (III) Supporting smooth collaborative interaction: This thrust will focus on the algorithm design that leverages the VR content similarities and predictabilities that naturally emerge during collaborative interactions. (IV) Scalable system integration, implementation, evaluation, and deployment: This thrust will integrate Research Thrusts I through III into a holistic system, perform system-level optimizations, and evaluate it through lab experiments and real-world classroom deployment.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

无线网络上的虚拟现实（VR）可以同时为多个用户提供交互式和沉浸式体验，因此具有许多应用，特别是在基于VR的教育/培训中。然而，这种无线沉浸式服务中令人满意的个性化用户体验需要严格的性能要求，包括：（1）高速和高分辨率的全景图像渲染;（2）极低的延迟保证;以及（3）无缝的用户体验。除了上述要求外，协同用户体验还需要VR服务的可扩展性和公平性。现有的VR系统严重依赖于各种启发式设计，并且没有有效地利用VR内容的共性及其可预测性，这阻碍了它们的大规模部署。该项目旨在开发一个通过无线网络提供个性化和可扩展的协作VR体验的系统的理论基础和完整实现。该项目将集成机器学习、无线网络和移动的计算，以在商用移动的设备上实现高质量和可扩展的无线沉浸式应用。本项目的理论和实际应用将被整合到本科和研究生课程中，并将让K-12学生接触到最先进的无线和VR技术。（2）用于多个用户的VR内容之间的通用性，以实现多播;以及（3）VR内容的可预测性，以实现预取。拟议的研究将有助于和推进无线网络和虚拟现实领域的理论和面向系统的研究。该项目明确利用沉浸式VR应用和无线网络的独特特性，并提出以下四个相互依赖的研究方向：（I）处理网络和预测不确定性：该方向将研究算法设计，以优化个性化用户体验，同时考虑网络和视口预测的不确定性。(II)满足严格的沉浸式服务要求：这一目标将开发无线调度算法，为多个VR用户提供严格的沉浸式、个性化服务保证。(III)支持流畅的协作交互：这一重点将集中在算法设计上，该设计利用了协作交互过程中自然出现的VR内容相似性和可预测性。(IV)可扩展的系统集成、实施、评估和部署：该项目将把研究重点I至III整合到一个整体系统中，进行系统级优化，并通过实验室实验和现实世界的课堂部署进行评估。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

项目成果

How Will VR Enter University Classrooms? Multi-stakeholders Investigation of VR in Higher Education

• DOI: 10.1145/3491102.3517542
• 发表时间: 2022-04
• 期刊: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems
• 作者: Qiao Jin;Yu Liu;S. Yarosh;Bo Han;Feng Qian

Vues: Practical Mobile Volumetric Video Streaming Through Multiview Transcoding

Vues：通过多视图转码实现实用的移动体积视频流

• DOI: 10.1145/3495243.3517027
• 发表时间: 2022
• 期刊: ACM MobiCom 2022
• 作者: Liu, Yu;Han, Bo;Qian, Feng;Narayanan, Arvind;Zhang, Zhi-Li
• 通讯作者: Zhang, Zhi-Li

Enhancing Quality of Experience for Collaborative Virtual Reality with Commodity Mobile Devices

• DOI: 10.1109/icdcs54860.2022.00102
• 发表时间: 2022-07
• 期刊: 2022 IEEE 42nd International Conference on Distributed Computing Systems (ICDCS)
• 作者: Jiangong Chen;Feng Qian;Bin Li

Collaborative Online Learning with VR Video: Roles of Collaborative Tools and Shared Video Control

• DOI: 10.1145/3544548.3581395
• 发表时间: 2023-04
• 期刊: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
• 作者: Qiao Jin;Yu Liu;Ruixuan Sun;Chen Chen-Chen;Puqi Zhou;Bo Han;Feng Qian;S. Yarosh