ICE-T: RC: Millimeter Wave Communications and Edge Computing for Next Generation Tetherless Mobile Virtual Reality

虚拟现实和增强现实(VR/AR)技术对我们的社会发展具有巨大的潜力，对生活质量、环境和能源节约以及世界经济都产生了影响。然而，两个主要挑战阻碍了这一愿景的实现。这些应用是超数据密集型的，并且需要超低延迟，这两个都不是当前和即将到来的传统联网方法和系统所能满足的。这些目前将VR/AR应用限制在离线操作、合成内容、高端有线设备和游戏/娱乐设置。该项目设想了一种将毫米波通信(MmWave)和边缘计算相结合的新型系统，旨在克服这些挑战，使我们更接近下一代无绳VR/AR社会应用。该项目将对网络VR/AR应用系统和通信的新兴领域做出显著贡献，导致许多与社会相关的应用程序取得进展，例如搜索和救援以及灾难应对。它还将促进新兴环境中大容量、高速/低延迟数据传输的一般应用领域的基础研究。除了直接的科学和技术影响及其对社会的更广泛影响之外，教育、外展、国际合作和科学领导活动将作为该项目的一个组成部分继续进行。克服目前和即将到来的联网系统能力和下一代应用的预期要求之间的巨大性能差距，将需要新的整体方法来捕获、编码、联网和重建/导航VR/AR数据。为此，该项目将研究一种未来的5G异质蜂窝网络系统，该系统集成了射频(RF)和毫米波通信，以及视点自适应时空可扩展VR信号拼接，用于360度无绳移动VR应用的多路径流传输。在这种背景下，该项目将进行以下协同调查：(1)导航感知可伸缩VR信号平铺，以实现仅在导航期间用户真正需要的数据的交互流传输；(2)用于用户导航预测的深度机器学习，以辅助设想的资源分配方法。(3)空时可伸缩的无码率码构造，用于有效的源-信道VR信号表示，以防止预期的传输错误。(4)针对射频-毫米波混合多路径VR流媒体的动态率失真优化策略，分析了VR信号拼接设计与两条网络路径特性之间相互依赖的基础。(5)分析边缘计算和通信之间的基本权衡，并寻求利用它们来最大化系统效率的优化方法。(6)动态毫米波发射机对VR用户分配问题的图论分析。(7)用于与其他应用程序并行操作的网络切片。将进行广泛的整合和实验，以在实际环境中评估、验证和原型启用的研究进展。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Virtual and augmented reality (VR/AR) technologies hold tremendous potential to advance our society, having impact on quality of life, environmental and energy conservation, and the world economy. However, two main challenges stand in the way of realizing this vision. These applications are hyper-data-intensive and require ultra-low latency, neither of which can be met by current and upcoming conventional networking methods and systems. These presently limit VR/AR applications to an offline operation, synthetic content, high-end wired equipment, and gaming/entertainment settings. This project envision a novel system at the intersection of millimeter-wave communication (mmWave) and edge computing that aims to overcome these challenges to bring us closer to the next generation tetherless VR/AR societal applications. The project will make notable contributions to the emerging area of networked VR/AR application systems and communications, leading to advances in numerous socially relevant applications, e.g., search and rescue, and disaster response. It will also facilitate fundamental research in the general application area of high-volume high-speed/low-latency data transfer in emerging settings. Beyond the direct scientific and technology impacts and their broader effects on society, educational, outreach, international collaboration, and scientific leadership activities will be pursued as an integral part of the project.Overcoming the broad performance gap between present and upcoming networked systems capabilities and anticipated requirements of next generation applications will require novel holistic approaches to capture, coding, networking, and reconstruction/navigation of VR/AR data. Towards this objective, the project will investigate a futuristic 5G heterogeneous cellular network system that integrates radio frequency (RF) and millimeter wave communication, and viewport-adaptive space-time scalable VR signal tiling, for multi-path streaming of 360-degree tetherless mobile VR applications. In this setting, the project will pursue the following synergistic investigations: (1) Navigation-aware scalable VR signal tiling to enable interactive streaming of only the data truly needed by the user during navigation; (2) Deep machine learning for user navigation prediction to assist the envisioned resource allocation methods. (3) Space-time scalable rateless code construction for effective source-channel VR signal representation to protect against prospective transmission errors. (4) Dynamic rate-distortion optimized strategies for hybrid RF-mmWave multi-path VR streaming and analysis of the foundations of the interdependencies between the VR signal tiling design and the characteristics of the two network paths. (5) Analysis of the fundamental trade-offs between edge computing and communication that arise here and pursuit of optimization methods that will leverage them to maximize the system efficiency. (6) Graph-theoretic analysis of the problem of dynamic mmWave transmitter to VR user assignment. (7) Network slicing for parallel operation with other applications. Extensive integration and experimentation will be carried out to assess, validate, and prototype the enabled research advances in practical settings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.