Towards field-of-view prediction for augmented reality applications on mobile devices
Towards field-of-view prediction for augmented reality applications on mobile devices
复制标题
DOI：
10.1145/3386293.3397114
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Na Wang;Haoliang Wang;Stefano Petrangeli;Viswanathan Swaminathan;Fei Li;Songqing Chen
中科院分区：
文献类型：
--
作者：
Na Wang;Haoliang Wang;Stefano Petrangeli;Viswanathan Swaminathan;Fei Li;Songqing Chen
通过允许人们操作放置在现实世界中的数字内容，增强现实（AR）在多个领域提供了沉浸式和丰富的体验。尽管它越来越受欢迎，但在带宽波动的情况下提供无缝的AR体验仍然是一个挑战，因为以逼真的质量且最小的延迟来提供这些体验需要高带宽。已经提出了流传输方法来解决这个问题，但它们需要准确预测用户的视野，以便仅流传输用户最有可能观看的场景区域。为了解决这个预测问题，我们在本文中研究了用户通过移动设备探索不同类型AR场景的观看行为。为此，我们引入了ACE数据集，这是第一个收集50名用户探索5种不同AR场景的移动数据的数据集。我们还为AR场景设计提出了一个包含四个特征的分类法，它能够以有条理的方式对不同类型的AR场景进行分类，并支持该领域的进一步研究。受ACE数据集分析结果的启发，我们开发了一种新的用户视觉注意力预测算法，该算法联合利用了用户历史移动信息和AR场景中数字对象的位置信息。在ACE数据集上的评估表明，所提出的方法在不同长度的预测范围内优于基线方法，因此在带宽降低和用户体验质量提高方面对AR生态系统有益。
By allowing people to manipulate digital content placed in the real world, Augmented Reality (AR) provides immersive and enriched experiences in a variety of domains. Despite its increasing popularity, providing a seamless AR experience under bandwidth fluctuations is still a challenge, since delivering these experiences at photorealistic quality with minimal latency requires high bandwidth. Streaming approaches have already been proposed to solve this problem, but they require accurate prediction of the Field-Of-View of the user to only stream those regions of scene that are most likely to be watched by the user. To solve this prediction problem, we study in this paper the watching behavior of users exploring different types of AR scenes via mobile devices. To this end, we introduce the ACE Dataset, the first dataset collecting movement data of 50 users exploring 5 different AR scenes. We also propose a four-feature taxonomy for AR scene design, which allows categorizing different types of AR scenes in a methodical way, and supporting further research in this domain. Motivated by the ACE dataset analysis results, we develop a novel user visual attention prediction algorithm that jointly utilizes information of users' historical movements and digital objects positions in the AR scene. The evaluation on the ACE Dataset show the proposed approach outperforms baseline approaches under prediction horizons of variable lengths, and can therefore be beneficial to the AR ecosystem in terms of bandwidth reduction and improved quality of users' experience.