CAREER: Networked Multi-User Augmented Reality for Mobile Devices

增强现实（AR）是一种新兴技术，在许多领域具有潜在的应用，包括教育，娱乐和公共安全。在单用户AR中，通过将虚拟对象叠加到真实的世界上来增强用户的感知。在多用户AR中，多个用户能够查看一组共同的虚拟对象并与之交互。然而，在当今的多用户AR应用中，由于AR应用缺乏先进的联网能力，用户可能会体验到较差的性能（例如高延迟或不同用户对虚拟对象的不一致视图）。该项目旨在通过开发底层增强现实平台的网络功能来解决用户体验问题，以支持未来的多用户增强现实应用。该项目的愿景是创建增强现实技术，使多个用户可以视觉扫描环境，并在现实世界的对象上快速弹出一组通用的相关虚拟对象。为了实现这一点，AR设备需要在它们之间有效地计算和交换信息，以便同步它们对真实的和虚拟世界的联合理解。计划中的研究有三个主要技术组成部分：（1）测量和表征多用户AR应用程序;（2）使AR应用程序适应计算和网络条件;（3）使网络适应AR应用程序的需求。评估场景包括教育AR课堂应用，以及5G测试平台中的移动的客户端。该项目的成功将在广泛的领域开辟一个新的多用户AR应用类别，包括教育（例如，多个配备AR的学生在一个教室里）和公共安全（例如，多个配备AR的第一响应者在灾难场景中）。该项目的研究结果将影响应用程序开发人员如何设计AR应用程序以及网络运营商如何管理AR网络流量。多用户AR平台将被纳入本科生课堂和校园创客空间，以及当地黑客松和高中研讨会的推广工作，旨在鼓励女性进入并留在工程领域。该项目的成果，包括论文，模拟和应用代码，以及AR质量评估和基准工具，将在项目网站https://github.com/multi-user-ar上提供。该网站和代码将在本项目期间提供和支持，并在此后在线保存。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Augmented reality (AR) is an emerging technology with potential applications in many fields, including education, entertainment, and public safety. In single-user AR, a user's perception is augmented by overlaying virtual objects onto the real world. In multi-user AR, multiple users are able to view and interact with a common set of virtual objects. However, in today's multi-user AR applications, users can experience poor performance (such as high latency or inconsistent views of the virtual objects by different users) due to the AR application's lack of advanced networking capabilities. This project seeks to address such user experience issues by developing the network capabilities of the underlying AR platform, in order to support future multi-user AR applications.The vision of this project is to create AR-enabling technology where multiple users can visually scan the environment, and a common set of relevant virtual objects swiftly pop up on top of the real-world objects. To accomplish this, the AR devices need to efficiently compute and exchange information amongst themselves, in order to synchronize their joint understanding of the real and virtual worlds. The planned research has three main technical components: (1) measuring and characterizing multi-user AR applications; (2) adapting an AR application to the computing and networking conditions; and (3) adapting the network to the needs of an AR application. Evaluation scenarios include an educational AR classroom application, as well as mobile clients in a 5G testbed.The success of this project will open up a new class of multi-user AR applications in a broad range of fields, including education (for example, multiple AR-equipped students in a classroom) and public safety (for example, multiple AR-equipped first responders in a disaster scenario). The findings from this project will impact how application developers design AR applications and how network operators manage AR network traffic. The multi-user AR platform will be incorporated into undergraduate classes and a campus makerspace, as well as in outreach efforts at local hackathons and high school workshops, with the aim of encouraging women to enter and stay in engineering fields.Results from this project, including papers, simulation and application code, and AR quality evaluation and benchmarking tools, will be available at the project website https://github.com/multi-user-ar. The website and code will be available and supported for the duration of this project, and preserved online thereafter.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.