ICE-T: RC: Millimeter Wave Communications and Edge Computing for Next Generation Tetherless Mobile Virtual Reality

虚拟现实和增强现实（VR/AR）技术具有巨大的潜力，可以推动我们的社会发展，对生活质量、环境和能源节约以及世界经济产生影响。然而，实现这一愿景的道路上存在两个主要挑战。这些应用程序是超数据密集型的，需要超低延迟，这是当前和即将到来的传统网络方法和系统无法满足的。目前，这些限制将VR/AR应用限制在离线操作、合成内容、高端有线设备和游戏/娱乐环境中。该项目设想在毫米波通信（mmWave）和边缘计算的交叉点上建立一个新的系统，旨在克服这些挑战，使我们更接近下一代无线VR/AR社会应用。该项目将为新兴的网络VR/AR应用系统和通信领域做出显著贡献，从而在许多社会相关应用领域取得进展，例如搜索和救援以及灾难响应。它还将促进新兴环境中大容量高速/低延迟数据传输的一般应用领域的基础研究。除了直接的科学和技术影响及其对社会的更广泛影响外，教育、推广、国际合作和科学领导活动将作为项目的一个组成部分进行。克服当前和即将到来的网络系统能力之间的巨大性能差距，以及下一代应用程序的预期需求，将需要新颖的整体方法来捕获、编码、联网和VR/AR数据的重建/导航。为了实现这一目标，该项目将研究未来的5G异构蜂窝网络系统，该系统集成了射频（RF）和毫米波通信，以及视口自适应时空可扩展的VR信号平铺，用于360度无线移动VR应用的多路径流。在这种情况下，该项目将进行以下协同研究：(1)导航感知可扩展的VR信号平铺，以实现用户在导航过程中真正需要的数据的交互流；(2)深度机器学习用户导航预测，辅助预想的资源分配方法。(3)时空可扩展的无速率编码结构，用于有效的源信道VR信号表示，以防止预期传输错误。(4)混合rf -毫米波多路径VR流的动态速率失真优化策略，并分析了VR信号平铺设计与两种网络路径特性相互依赖的基础。(5)分析这里出现的边缘计算和通信之间的基本权衡，并寻求优化方法，利用它们最大化系统效率。(6)动态毫米波发射机对VR用户分配问题的图论分析。(7)与其他应用程序并行操作的网络切片。将进行广泛的整合和实验，以评估、验证和在实际环境中实现研究进展的原型。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Virtual and augmented reality (VR/AR) technologies hold tremendous potential to advance our society, having impact on quality of life, environmental and energy conservation, and the world economy. However, two main challenges stand in the way of realizing this vision. These applications are hyper-data-intensive and require ultra-low latency, neither of which can be met by current and upcoming conventional networking methods and systems. These presently limit VR/AR applications to an offline operation, synthetic content, high-end wired equipment, and gaming/entertainment settings. This project envision a novel system at the intersection of millimeter-wave communication (mmWave) and edge computing that aims to overcome these challenges to bring us closer to the next generation tetherless VR/AR societal applications. The project will make notable contributions to the emerging area of networked VR/AR application systems and communications, leading to advances in numerous socially relevant applications, e.g., search and rescue, and disaster response. It will also facilitate fundamental research in the general application area of high-volume high-speed/low-latency data transfer in emerging settings. Beyond the direct scientific and technology impacts and their broader effects on society, educational, outreach, international collaboration, and scientific leadership activities will be pursued as an integral part of the project.Overcoming the broad performance gap between present and upcoming networked systems capabilities and anticipated requirements of next generation applications will require novel holistic approaches to capture, coding, networking, and reconstruction/navigation of VR/AR data. Towards this objective, the project will investigate a futuristic 5G heterogeneous cellular network system that integrates radio frequency (RF) and millimeter wave communication, and viewport-adaptive space-time scalable VR signal tiling, for multi-path streaming of 360-degree tetherless mobile VR applications. In this setting, the project will pursue the following synergistic investigations: (1) Navigation-aware scalable VR signal tiling to enable interactive streaming of only the data truly needed by the user during navigation; (2) Deep machine learning for user navigation prediction to assist the envisioned resource allocation methods. (3) Space-time scalable rateless code construction for effective source-channel VR signal representation to protect against prospective transmission errors. (4) Dynamic rate-distortion optimized strategies for hybrid RF-mmWave multi-path VR streaming and analysis of the foundations of the interdependencies between the VR signal tiling design and the characteristics of the two network paths. (5) Analysis of the fundamental trade-offs between edge computing and communication that arise here and pursuit of optimization methods that will leverage them to maximize the system efficiency. (6) Graph-theoretic analysis of the problem of dynamic mmWave transmitter to VR user assignment. (7) Network slicing for parallel operation with other applications. Extensive integration and experimentation will be carried out to assess, validate, and prototype the enabled research advances in practical settings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.