Augmenting Augmented Reality with Non-Line-of-Sight Perception
Augmenting Augmented Reality with Non-Line-of-Sight Perception
复制标题
DOI：
--
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Tara Boroushaki;Maisy Lam;L. Dodds;A. Eid;Fadel M. Adib
中科院分区：
文献类型：
--
作者：
Tara Boroushaki;Maisy Lam;L. Dodds;A. Eid;Fadel M. Adib
-我们介绍了X-AR的设计、实现和评估，这是一个具有非视线感知的增强现实(AR)系统。X-AR耳机增加了射频感应，使用户能够看到人眼或最先进的AR系统看不到的东西。我们的设计引入了三个主要创新：第一个是AR共形天线，它与AR耳机护罩的形状紧密匹配，同时为射频传感提供出色的辐射和带宽能力。第二种是射频视觉合成孔径定位算法，它利用人的自然移动性在视线和非视线环境中定位射频标记的对象。最后，第三个是RF-视觉验证原语，它融合了RF和VISION，向最终用户提供可操作的任务，如挑选验证。我们通过将其集成到Microsoft Hololens 2 AR耳机中，构建了我们设计的端到端原型，并在视线和非视线环境中对其进行了评估。实验结果表明，X-AR实现了完全遮挡物品的分米级射频定位(中位数为9.8厘米)，在提取RFID标签物品时，可以进行射频视觉拾取验证，准确率超过95%(F-Score)。这些结果表明，X-AR成功地将AR系统扩展到非视距感知，对制造、仓储和智能家居应用具有重要意义。演示视频：y2U.S.be/bdUN21ft7G0
– We present the design, implementation, and evaluation of X-AR, an augmented reality (AR) system with non-line-of-sight perception. X-AR augments AR headsets with RF sensing to enable users to see things that are otherwise invisible to the human eye or to state-of-the-art AR systems. Our design introduces three main innovations: the first is an AR-conformal antenna that tightly matches the shape of the AR headset visor while providing excellent radiation and bandwidth capabilities for RF sensing. The second is an RF-visual synthetic aperture localization algorithm that leverages natural human mobility to localize RF-tagged objects in line-of-sight and non-line-of-sight settings. Finally, the third is an RF-visual verification primitive that fuses RF and vision to deliver actionable tasks to end users such as picking verification. We built an end-to-end prototype of our design by integrating it into a Microsoft Hololens 2 AR headset and evaluated it in line-of-sight and non-line-of-sight environments. Our results demonstrate that X-AR achieves decimeter-level RF localization (median of 9.8 cm) of fully-occluded items and can perform RF-visual picking verification with over 95% accuracy (F-Score) when extracting RFID-tagged items. These re-sults show that X-AR is successful in extending AR systems to non-line-of-sight perception, with important implications to manufacturing, warehousing, and smart home applications. Demo video: y2u.be/bdUN21ft7G0