RINGS: l-RIM: Learning based Resilient Immersive Media-Compression, Delivery, and Interaction

增强现实/虚拟现实（AR/VR）已被公认为下一代（NextG）网络系统的变革性服务，而无线支持的AR/VR将为用户提供极大的灵活性和增强的沉浸式体验，并释放出大量的新应用。该项目的主要目标是探索NextG和人工智能（AI）中的创新技术，以提供统一的媒体压缩，通信和计算框架，从而实现弹性无线AR/VR。研究议程包括五个重点。重点1和重点2关注基于学习的沉浸式媒体压缩，即，分别开发高效的光场和点云压缩解决方案。接下来的两个重点将探讨促进无线AR/VR传输和交互的基本概念和技术。Thrust 3将开发理论模型，并探索无线AR/VR的压缩通信，计算和缓存之间的基本性能权衡，考虑到几种NextG通信和网络技术。Thrust 4的工作将研究边缘智能的应用，以开发基于学习的算法，用于弹性AR/VR传输。弹性AR/VR的设计将以Thrust 3中开发的基于模型的性能分析为指导。 Thrust 5将整合Thrusts 1-4中开发的技术，并通过使用开源数据集的模拟研究和使用AR/VR测试平台以及公开可用的无线和云相关平台的实验研究来验证其性能。拟议的研究具有很高的潜力，可以在我们的研究社区和社会中产生巨大的影响。无线社区一直在努力为NextG无线和移动的通信、联网、传感和计算系统开发基于学习的解决方案。该项目的研究结果有望促进实现沉浸式媒体应用程序的突破，这些应用程序需要大量数据来训练复杂的深度学习模型。PI致力于研究和教育的整合。该项目将为两个合作机构的本科生和研究生提供研究和教育机会。将开发新的培训证书课程，研究成果将纳入即将出版的PI Mao教科书。这两个机构现有的几个外联方案将被用来扩大代表性不足的群体的参与。研究成果将通过学术出版物、项目网站、开放源码储存库以及由项目执行机构组织的讲习班和期刊特刊传播。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Augmented Reality/Virtual Reality (AR/VR) has been recognized as a transformative service for the Next Generation (NextG) network systems, while wireless supported AR/VR will offer great flexibility and enhanced immersive experience to users and unleash a plethora of new applications. The primary goal of this project is to explore innovative technologies in NextG and Artificial intelligence (AI) to provide a unified media compression, communication, and computing framework to enable resilient wireless AR/VR. The research agenda comprises five thrusts. Thrusts 1 and 2 are focused on learning-based immersive media compression, i.e., developing high-efficiency Light Field and Point Cloud compression solutions, respectively. The next two thrusts explore the fundamental concepts and techniques that facilitate wireless AR/VR transmission and interaction. Thrust 3 shall develop theoretical models and explore the fundamental performance tradeoffs among compression communication, computation, and caching for wireless AR/VR considering several NextG communications and networking technologies. The work in Thrust 4 shall investigate the application of edge intelligence to develop learning-based algorithms for resilient AR/VR transmissions. The design of resilient AR/VR will be guided by the model-based performance analysis developed in Thrust 3. Thrust 5 will integrate the techniques developed in Thrusts 1-4, and validate their performance with simulation studies using open-source datasets and experimental studies using an AR/VR testbed and publicly available wireless and cloud-related platforms.The proposed research has a high potential to achieve great impacts in our research community and society. The wireless community has been making great efforts on developing learning-based solutions for the NextG wireless and mobile communication, networking, sensing, and computing systems. Findings from this project are expected to facilitate breakthroughs in enabling immersive media applications that require huge amounts of data to train complex deep learning models. The PIs are committed to integration of research and education. This project will offer research and educational opportunities for undergraduate and graduate students at the two collaborating institutions. New training certificate courses will be developed and research outcomes will be incorporated into a forthcoming textbook by PI Mao. The several existing outreach programs at the two institutions will be leveraged to broaden participation from underrepresented groups. Research outcomes will be disseminated through scholarly publications, a project website, open-source repositories, as well as workshops and journal special issues to be organized by the PIs. Industry will also be pursued through the NSF IUCRC Center for Big Learning at The University of Missouri, Kansas City.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.