Collaborative Research: CCRI: New: CoMIC: A Collaborative Mobile Immersive Computing Research Infrastructure for Multi-user XR

协作研究：CCRI：新：CoMIC：用于多用户 XR 的协作移动沉浸式计算研究基础设施

• 批准号: 2235049
• 负责人: Bo Han
• 金额: $ 150万
• 依托单位: George Mason University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-05-15 至 2026-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2235049&HistoricalAwards=false
• 关键词: Collaborative; Research; CCRI; New; CoMIC

The last few years have witnessed the tremendous development of immersive technologies such as virtual, augmented, and mixed reality (VR, AR, and MR), which are often collectively referred to as extended reality (XR). While XR is deemed as the killer application of 5G and beyond, existing work has mainly focused on improving the quality of experience (QoE) for single-user scenarios, and there is limited research on exploring the underlying infrastructure to support emerging immersive applications, especially multi-user collaboration and interaction involving participants from geo-distributed locations. This project aims to design and develop the infrastructure for Collaborative Mobile Immersive Computing (CoMIC), which fills this critical gap and lays the technical foundation for realizing the grand vision of the Metaverse. The intellectual merit of CoMIC includes: (1) constructing a research infrastructure with both software and hardware components for advancing collaborative immersive computing and (2) serving the community by empowering innovative solutions of multi-user XR. CoMIC will develop a first-of-its-kind, open-accessible, and reconfigurable research infrastructure, which is naturally required by various use cases such as education, training, healthcare, and entertainment. It strives to offer a collaborative, visual-first, and hologram-based computing space and a QoE-driven, multi-site, and immersive communication framework by providing a shared environment for community users. CoMIC will be open to and support in-person visits of community users to facilitate their research projects. This project will create broader impacts on research infrastructure, education and training, societies, and economies, and broaden computing participation. Currently, the research and development of multi-user XR have been hindered because the underlying infrastructure with essential building blocks is still missing. CoMIC will timely bridge this critical gap and significantly speed up the realization of the Metaverse vision. The CoMIC platform offers a unique research opportunity for students and will be used to educate and train the next generation of researchers and engineers to master the key technologies and system design skills in immersive computing and advance the frontiers of science and engineering. This collaborative project brings together investigators from George Mason University and the University of Maryland to build the CoMIC infrastructure. Its remotely accessible and programmable robots mounted with XR devices and mmWave 5G edge networks can enable the comparison of different solutions in a consistent way. Users can download the CoMIC software to set up their own testbed, replace the CoMIC components in the reference implementation, and experiment with their own multi-user XR solutions. The website for this project is https://comic-xr.github.io/This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

近几年来，虚拟现实、增强现实和混合现实（VR、AR和MR）等沉浸式技术取得了巨大的发展，这些技术通常被统称为延展实境（XR）。虽然XR被认为是5G及以后的杀手级应用，但现有的工作主要集中在提高单用户场景的体验质量（QoE）上，并且在探索底层基础设施以支持新兴沉浸式应用方面的研究有限，特别是涉及地理分布位置的参与者的多用户协作和交互。该项目旨在设计和开发协作移动的沉浸式计算（CoMIC）的基础设施，填补这一关键空白，为实现Metaverse的宏伟愿景奠定技术基础。CoMIC的智力价值包括：（1）构建具有软件和硬件组件的研究基础设施，以推进协作沉浸式计算;（2）通过支持多用户XR的创新解决方案来服务社区。CoMIC将开发一个首创的、开放访问的、可重新配置的研究基础设施，这是教育、培训、医疗保健和娱乐等各种用例所自然需要的。它致力于通过为社区用户提供共享环境，提供协作，视觉优先和基于全息图的计算空间以及QoE驱动的，多站点和沉浸式通信框架。COMIC将开放并支持社区用户的亲自访问，以促进他们的研究项目。该项目将对研究基础设施、教育和培训、社会和经济产生更广泛的影响，并扩大计算参与。目前，多用户XR的研究和开发受到阻碍，因为仍然缺少具有基本构建块的底层基础设施。CoMIC将及时弥合这一关键差距，并显著加快Metaverse愿景的实现。CoMIC平台为学生提供了一个独特的研究机会，并将用于教育和培训下一代研究人员和工程师，以掌握沉浸式计算的关键技术和系统设计技能，并推进科学和工程的前沿。这个合作项目汇集了来自乔治梅森大学和马里兰州大学的研究人员，以建立COMIC基础设施。其安装有XR设备和mmWave 5G边缘网络的可远程访问和可编程机器人可以以一致的方式比较不同的解决方案。用户可以下载CoMIC软件来建立自己的测试平台，替换参考实现中的CoMIC组件，并使用自己的多用户XR解决方案进行实验。该项目的网站是https://comic-xr.github.io/This奖项反映了NSF的法定使命，并已被认为值得通过使用基金会的知识价值和更广泛的影响审查标准进行评估的支持。

项目成果

Demystifying Web-based Mobile Extended Reality Accelerated by WebAssembly

• DOI: 10.1145/3618257.3624833
• 发表时间: 2023-10
• 期刊: Proceedings of the 2023 ACM on Internet Measurement Conference
• 作者: Kaiyan Liu;Nan Wu;Bo Han

Environment-driven mmWave Beamforming for Multi-user Immersive Applications

• DOI: 10.1145/3615452.3617945
• 发表时间: 2023-10
• 期刊: Proceedings of the 1st ACM Workshop on Mobile Immersive Computing, Networking, and Systems
• 作者: Ahmad Kamari;Nan Wu;Hemant Kumar Narsani;Bo Han;Parth Pathak

Toward Next-generation Volumetric Video Streaming with Neural-based Content Representations

• DOI: 10.1145/3615452.3617938
• 发表时间: 2023-10
• 期刊: Proceedings of the 1st ACM Workshop on Mobile Immersive Computing, Networking, and Systems

Enriching Telepresence with Semantic-driven Holographic Communication

• DOI: 10.1145/3626111.3628184
• 发表时间: 2023-11
• 期刊: Proceedings of the 22nd ACM Workshop on Hot Topics in Networks
• 作者: Ruizhi Cheng;Kaiyan Liu;Nan Wu;Bo Han