CAREER: Networked Multi-User Augmented Reality for Mobile Devices

增强现实（AR）是一项新兴技术，在许多领域都有潜在的应用，包括教育、娱乐和公共安全。在单用户增强现实中，用户的感知通过将虚拟物体叠加到现实世界中来增强。在多用户AR中，多个用户能够查看一组共同的虚拟对象并与之交互。然而，在当今的多用户AR应用程序中，由于AR应用程序缺乏先进的网络功能，用户可能会体验到较差的性能（例如高延迟或不同用户对虚拟对象的视图不一致）。该项目旨在通过开发底层AR平台的网络功能来解决此类用户体验问题，以支持未来的多用户AR应用。该项目的愿景是创建增强现实技术，让多个用户可以直观地扫描环境，一组通用的相关虚拟物体迅速出现在现实世界的物体之上。为了实现这一目标，AR设备需要有效地计算和交换信息，以同步它们对真实世界和虚拟世界的共同理解。计划中的研究有三个主要技术组成部分：(1)测量和表征多用户AR应用；(2)使增强现实应用适应计算和网络条件；(3)使网络适应增强现实应用的需求。评估场景包括一个教育AR教室应用，以及5G测试平台上的移动客户端。该项目的成功将在广泛的领域开辟一类新的多用户AR应用，包括教育（例如，教室中有多个配备AR的学生）和公共安全（例如，灾难场景中有多个配备AR的第一响应者）。该项目的研究结果将影响应用程序开发人员如何设计AR应用程序以及网络运营商如何管理AR网络流量。多用户AR平台将被纳入本科课程和校园创客空间，以及当地黑客马拉松和高中研讨会的推广工作，目的是鼓励女性进入并留在工程领域。该项目的成果，包括论文、模拟和应用代码，以及AR质量评估和基准测试工具，将在项目网站https://github.com/multi-user-ar上提供。网站和代码将在项目期间提供支持，并在项目结束后在线保存。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Augmented reality (AR) is an emerging technology with potential applications in many fields, including education, entertainment, and public safety. In single-user AR, a user's perception is augmented by overlaying virtual objects onto the real world. In multi-user AR, multiple users are able to view and interact with a common set of virtual objects. However, in today's multi-user AR applications, users can experience poor performance (such as high latency or inconsistent views of the virtual objects by different users) due to the AR application's lack of advanced networking capabilities. This project seeks to address such user experience issues by developing the network capabilities of the underlying AR platform, in order to support future multi-user AR applications.The vision of this project is to create AR-enabling technology where multiple users can visually scan the environment, and a common set of relevant virtual objects swiftly pop up on top of the real-world objects. To accomplish this, the AR devices need to efficiently compute and exchange information amongst themselves, in order to synchronize their joint understanding of the real and virtual worlds. The planned research has three main technical components: (1) measuring and characterizing multi-user AR applications; (2) adapting an AR application to the computing and networking conditions; and (3) adapting the network to the needs of an AR application. Evaluation scenarios include an educational AR classroom application, as well as mobile clients in a 5G testbed.The success of this project will open up a new class of multi-user AR applications in a broad range of fields, including education (for example, multiple AR-equipped students in a classroom) and public safety (for example, multiple AR-equipped first responders in a disaster scenario). The findings from this project will impact how application developers design AR applications and how network operators manage AR network traffic. The multi-user AR platform will be incorporated into undergraduate classes and a campus makerspace, as well as in outreach efforts at local hackathons and high school workshops, with the aim of encouraging women to enter and stay in engineering fields.Results from this project, including papers, simulation and application code, and AR quality evaluation and benchmarking tools, will be available at the project website https://github.com/multi-user-ar. The website and code will be available and supported for the duration of this project, and preserved online thereafter.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.