Collaborative Research: CSR: Medium: Adaptive Environmental Awareness for Collaborative Augmented Reality

协作研究：企业社会责任：媒介：协作增强现实的自适应环境意识

• 批准号: 2312760
• 负责人: Maria Gorlatova
• 金额: $ 40万
• 依托单位: Duke University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2027-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2312760&HistoricalAwards=false
• 关键词: Collaborative; Research; CSR; Medium; Adaptive

Augmented reality (AR) represents an exciting new frontier for mobile applications, merging the physical and virtual worlds to facilitate real-time interactions between users and their surrounding physical environments. Next-generation AR is centered on interactive and environmentally-aware applications, which sense the state of the real world via computationally heavy perception algorithms, with results augmented in real-time for multiple users. However, current environmentally-aware AR applications are well-recognized to lack sufficient resilience for long-term, large-scale, and safety-critical deployments. The goal of this project is to exploit the collaborative nature of next-generation AR applications to provide reliable, real-time environmental awareness that can adapt to the surrounding environment, user mobility, and computing resource availability. The core insight of this project is that instead of simply competing with one another for network and compute resources, AR devices can collaborate with one another to run and adapt AR applications to optimize overall utility. The project will involve modeling environmental awareness and mobility patterns in AR, incorporating these models into an emulator to facilitate experimentation and training, and designing reinforcement learning policies that capture environmental awareness and dynamic user relationships. These policies will be deployed in a set of new collaborative AR applications, such as AR-based physical rehabilitation, and evaluated via expansive studies with multiple groups of users in a wide range of conditions. This project will lay the groundwork for resilient next-generation AR applications, which can benefit multiple domains, including retail, manufacturing, and healthcare. The project will develop an example next-generation AR-supported rehabilitation application, which will be showcased to clinicians across multiple hospital systems and will be integrated into the PIs’ K-12 and public outreach efforts. The multi-user AR emulator created as part of this project will be open-sourced to enable the development of a wide range of other applications and will be designed to be extensible to additional use cases, such as AR-aided human-robot collaboration. Moreover, the machine learning techniques for dynamic user collaboration developed as part of this project will be applicable to other cases in which multiple entities interact dynamically in a coordinated manner, such as supply chain management. The project will train several cohorts of undergraduate students, and results will be presented at multiple events and venues aimed at broadening participation in computing.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

增强现实（AR）代表了移动应用程序的一个令人兴奋的新领域，将物理和虚拟世界融合在一起，以促进用户及其周围物理环境之间的实时互动。下一代AR以交互式和环境意识的应用程序为中心，这些应用程序通过计算沉重的感知算法感知现实世界的状态，并对多个用户实时增强结果。但是，当前对环境意识的AR应用已被众所周知，缺乏足够的弹性，对于长期，大规模和关键安全部署。该项目的目的是探索下一代AR应用程序的协作性质，以提供可靠的实时环境意识，以适应周围环境，用户移动性和计算资源可用性。该项目的核心洞察力是，AR设备可以相互协作以运行和调整AR应用程序以优化整体实用程序，而不是简单地互相竞争网络和计算资源。该项目将涉及对AR中的环境意识和移动性模式进行建模，并将这些模型纳入模拟器中，以促进实验和培训，并设计强化学习政策，以捕捉环境意识和动态用户关系。这些政策将部署在一系列新的协作AR应用程序中，例如基于AR的物理康复，并通过与多种情况下的多种用户进行额外研究进行评估。该项目将为累积的下一代AR应用奠定基础，这些应用程序可以使包括零售，制造业和医疗保健在内的多个领域受益。该项目将开发一个示例下一代AR支持的康复应用程序，该应用程序将在多个医院系统的临床医生中展示，并将纳入PIS的K-12和公共外展工作。作为该项目的一部分而创建的多用户AR模拟器将被开源以实现广泛的其他应用程序的开发，并将设计为广泛的用例，例如AR-ADED人类机器人协作。此外，作为该项目的一部分开发的动态用户协作的机器学习技术将适用于多个实体以协调方式（例如供应链管理）动态交互的其他情况。该项目将培训几个本科生，并将在旨在扩大计算参与的多个活动和场地上提出结果。该奖项反映了NSF的法定任务，并通过使用该基金会的知识分子的智力优点和更广泛的影响来评估NSF的法定任务。