CNS Core: Small: Collaborative Research: Towards Intelligent Multi-User Augmented Reality with Edge Computing

增强现实(AR)将数字内容覆盖在用户周围的现实世界中，直到最近才对日常用户可用。AR应用程序在包括教育、医疗和游戏在内的多个领域得到了初步采用。然而，这项技术目前还处于初级阶段，其适应用户偏好和环境条件的能力有限，提供的多用户能力也有限。提供自适应多用户AR体验对现有的移动系统和自适应算法提出了挑战。从移动系统的角度来看，AR设备有严格的能源和计算限制，可能会遇到网络故障。在算法上，现有的自适应算法往往是基于分布式机器学习的，而分布式机器学习并不是被设计成在受限的AR设备上运行的，并且可能不能很好地应对环境动态。这项工作将利用边缘计算作为移动系统挑战的解决方案，设计优化的系统架构，以提供多用户AR所需的相似但不相同的用户体验。这项工作还将量化现有分布式学习方法在多用户AR约束下的性能，并将开发优化由此产生的性能权衡的算法。这项工作最终将产生与AR相关的输入、输出以及系统和网络资源利用特征的数据集，该数据集将公开发布，供更广泛的研究人员使用，以验证他们在AR系统和算法方面的工作。该项目将实现现有单机或云支持的AR无法实现的AR功能，并将扩大机器学习在AR中的应用范围。这项工作将允许开发用于教育、医疗、零售和游戏的新AR应用程序，通过提供更个性化和动态的用户体验来更好地吸引用户。开发的解决方案将在杜克狐猴中心的特定教育环境中进行测试，AR将用于教育该中心的参观者关于濒危动物和保护工作的知识。这项研究将被整合到杜克大学和卡内基梅隆大学的课程中，并将被用作几个本科生和研究生独立研究项目的基础。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Augmented reality (AR), which overlays digital content with the real world around a user, has only recently become available to everyday users. AR applications are finding initial adoption in multiple areas including education, medicine, and gaming. Yet, the technology is currently in its infancy, limited in its ability to adapt to user preferences and environmental conditions, and offering restricted multi-user capabilities. Delivering adaptive multi-user AR experiences poses challenges to existing mobile systems and adaptation algorithms. From a mobile systems perspective, AR devices have strict energy and computing constraints and may experience network failures. Algorithmically, existing adaptation algorithms are often based on distributed machine learning, which is not designed to run on constrained AR devices and may not cope well with environment dynamics. This work will leverage edge computing as a solution for the mobile systems challenges, designing system architectures optimized to provide the similar but not identical user experiences required for multi-user AR. The work will also quantify the performance of existing distributed learning approaches under the constraints of multi-user AR, and will develop algorithms that optimize the resulting performance tradeoffs. The work will finally result in a dataset of AR-related inputs, outputs, and system and network resource utilization characteristics, which will be released publicly for the use of a wider community of researchers to validate their work on AR systems and algorithms. This project will enable AR capabilities not possible with existing stand-alone or cloud-supported AR, and will extend the scope of applications of machine learning in AR. The work will allow for the development of new AR applications for education, medicine, retail, and gaming that better engage users through delivering more personalized and dynamic user experiences. The developed solutions will be tested in education-specific settings at Duke Lemur Center, where AR will be used to educate visitors to the center about endangered animals and conservation efforts. The research will be integrated into course curricula at Duke and Carnegie Mellon and will be used as the basis for several undergraduate and graduate independent research projects.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.