Enriching Telepresence with Semantic-driven Holographic Communication
Enriching Telepresence with Semantic-driven Holographic Communication
复制标题
DOI：
10.1145/3626111.3628184
复制
发表时间：
2023-11
期刊：
影响因子：
--
通讯作者：
Ruizhi Cheng;Kaiyan Liu;Nan Wu;Bo Han
中科院分区：
文献类型：
--
作者：
Ruizhi Cheng;Kaiyan Liu;Nan Wu;Bo Han
实现最小化带宽消耗和端到端延迟的最佳平衡，同时保持令人满意的视觉质量水平，成为实时交互式全息通信的最终目标，这是6G沉浸式远程呈现的基本构建块。然而，考虑到要流传输的大量3D数据、苛刻的延迟要求以及所涉及的高计算工作量，实现这一雄心勃勃的目标对具有有限计算能力的移动的设备提出了重大挑战。而不是分发沉浸式的内容一点一点，在这个立场文件中，我们建议提供语义信息提取的网真参与者，以大幅减少互联网带宽的使用面向任务的应用程序，如远程协作。我们通过将相关语义分类为三种不同类型（即，关键点，2D图像和文本），明确了与为我们全面的研究议程中的每个类别开发实用系统相关的开放研究挑战，并深入研究克服这些挑战的潜在解决方案。我们的概念验证实现的初步结果，利用基于关键点的语义（部分）验证了我们的研究议程的可行性。
Achieving the optimal balance of minimizing bandwidth consumption and end-to-end latency while preserving a satisfactory level of visual quality becomes the ultimate goal of live, interactive holographic communication, a fundamental building block of immersive telepresence envisioned for 6G. Nevertheless, achieving this ambitious goal poses significant challenges for mobile devices with limited computing power, considering the substantial amount of 3D data to stream, the demanding latency requirements, and the high computation workload involved. Instead of distributing immersive content bit by bit, in this position paper, we propose to deliver semantic information extracted from telepresence participants to drastically reduce Internet bandwidth usage for task-oriented applications such as remote collaboration. We contribute a taxonomy by categorizing related semantics into three different types (i.e., keypoints, 2D images, and text), pinpoint the open research challenges associated with developing a practical system for each category in our comprehensive research agenda, and delve into the potential solutions for overcoming these challenges. The preliminary results from our proof-of-concept implementation that harnesses keypoint-based semantics (partially) validate the feasibility of our research agenda.