ARCHIE++ : A Cloud-enabled Framework for Conducting AR System Testing in the Wild
ARCHIE++ : A Cloud-enabled Framework for Conducting AR System Testing in the Wild
复制标题
ARCHIE：用于在野外进行 AR 系统测试的云支持框架
DOI：
10.1109/tvcg.2022.3141029
复制
发表时间：
2022
影响因子：
5.2
通讯作者：
Tan, Chiu
中科院分区：
文献类型：
--
作者：
Lehman, Sarah;Elezovikj, Semir;Ling, Haibin;Tan, Chiu
在本文中，我们提出了ARCHIE++，一个测试框架进行AR系统测试和收集用户反馈在野外。我们的系统通过将可用性反馈数据（现场收集）与同一时间段的系统性能数据相结合，解决了AR测试实践中的挑战。然后，这些数据包可以被用来识别测试人员在不受约束的使用场景中遇到的边缘情况。我们开始提出了一组当前的趋势，在执行AR系统的人体测试，通过审查最近的工作选择从混合现实，人为因素，和移动的和普及系统。从这些趋势中，我们确定了在尝试采用这些做法进行野外测试时所面临的一系列挑战。这些挑战被用来为我们的框架设计提供信息，该框架为AR系统开发人员提供了一种支持云和设备不可知的方式，以提高他们对环境条件的了解，并在野外测试时支持可扩展性和可再现性。然后，我们提出了一系列的案例研究，展示了如何ARCHIE++可以用来支持一系列的AR测试场景，并通过一系列的评估证明了框架的有限开销。最后，我们对ARCHIE++在各种边缘条件下的设计和实用性进行了更多讨论。
In this paper, we present ARCHIE++, a testing framework for conducting AR system testing and collecting user feedback in the wild. Our system addresses challenges in AR testing practices by aggregating usability feedback data (collectedin situ) with system performance data from that same time period. These data packets can then be leveraged to identify edge cases encountered by testers during unconstrained usage scenarios. We begin by presenting a set of current trends in performing human testing of AR systems, identified by reviewing a selection of recent work from leading conferences in mixed reality, human factors, and mobile and pervasive systems. From the trends, we identify a set of challenges to be faced when attempting to adopt these practices to testing in the wild. These challenges are used to inform the design of our framework, which provides a cloud-enabled and device-agnostic way for AR systems developers to improve their knowledge of environmental conditions and to support scalability and reproducibility when testing in the wild. We then present a series of case studies demonstrating how ARCHIE++ can be used to support a range of AR testing scenarios, and demonstrate the limited overhead of the framework through a series of evaluations. We close with additional discussion on the design and utility of ARCHIE++ under various edge conditions.