CNS Core: Small: Collaborative Research: Towards Intelligent Multi-User Augmented Reality with Edge Computing

CNS 核心：小型：协作研究：利用边缘计算实现智能多用户增强现实

• 批准号: 1909306
• 负责人: Carlee Joe-Wong
• 金额: $ 25万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1909306&HistoricalAwards=false
• 关键词: CNS; Core; Small; Collaborative; Research

Augmented reality (AR), which overlays digital content with the real world around a user, has only recently become available to everyday users. AR applications are finding initial adoption in multiple areas including education, medicine, and gaming. Yet, the technology is currently in its infancy, limited in its ability to adapt to user preferences and environmental conditions, and offering restricted multi-user capabilities. Delivering adaptive multi-user AR experiences poses challenges to existing mobile systems and adaptation algorithms. From a mobile systems perspective, AR devices have strict energy and computing constraints and may experience network failures. Algorithmically, existing adaptation algorithms are often based on distributed machine learning, which is not designed to run on constrained AR devices and may not cope well with environment dynamics. This work will leverage edge computing as a solution for the mobile systems challenges, designing system architectures optimized to provide the similar but not identical user experiences required for multi-user AR. The work will also quantify the performance of existing distributed learning approaches under the constraints of multi-user AR, and will develop algorithms that optimize the resulting performance tradeoffs. The work will finally result in a dataset of AR-related inputs, outputs, and system and network resource utilization characteristics, which will be released publicly for the use of a wider community of researchers to validate their work on AR systems and algorithms. This project will enable AR capabilities not possible with existing stand-alone or cloud-supported AR, and will extend the scope of applications of machine learning in AR. The work will allow for the development of new AR applications for education, medicine, retail, and gaming that better engage users through delivering more personalized and dynamic user experiences. The developed solutions will be tested in education-specific settings at Duke Lemur Center, where AR will be used to educate visitors to the center about endangered animals and conservation efforts. The research will be integrated into course curricula at Duke and Carnegie Mellon and will be used as the basis for several undergraduate and graduate independent research projects.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

将数字内容与用户周围的真实的世界叠加的增强现实（AR）最近才变得可用于日常用户。AR应用程序在多个领域得到初步采用，包括教育，医疗和游戏。然而，该技术目前处于起步阶段，适应用户偏好和环境条件的能力有限，并且提供有限的多用户能力。提供自适应多用户AR体验对现有的移动的系统和自适应算法提出了挑战。从移动的系统的角度来看，AR设备具有严格的能量和计算约束，并且可能经历网络故障。从理论上讲，现有的自适应算法通常基于分布式机器学习，而分布式机器学习并不是为在受限的AR设备上运行而设计的，并且可能无法很好地科普环境动态。这项工作将利用边缘计算作为移动的系统挑战的解决方案，设计优化的系统架构，以提供多用户AR所需的相似但不相同的用户体验。这项工作还将量化现有分布式学习方法在多用户AR约束下的性能，并将开发优化由此产生的性能权衡的算法。这项工作最终将产生一个与AR相关的输入、输出以及系统和网络资源利用特征的数据集，该数据集将公开发布，供更广泛的研究人员社区使用，以验证他们在AR系统和算法方面的工作。该项目将实现现有独立或云支持的AR无法实现的AR功能，并将扩展机器学习在AR中的应用范围。这项工作将允许开发用于教育，医疗，零售和游戏的新AR应用程序，通过提供更个性化和动态的用户体验来更好地吸引用户。开发的解决方案将在杜克狐猴中心的特定教育环境中进行测试，在那里AR将用于教育中心的游客了解濒危动物和保护工作。该研究将被纳入课程课程在杜克和卡内基梅隆大学，并将被用作基础上的几个本科生和研究生的独立研究项目。这个奖项反映了NSF的法定使命，并已被认为是值得通过评估使用基金会的智力价值和更广泛的影响审查标准的支持。

项目成果

Towards Flexible Device Participation in Federated Learning

实现联邦学习中灵活的设备参与

• 发表时间: 2021
• 期刊: Proceedings of The 24th International Conference on Artificial Intelligence and Statistics
• 作者: Ruan, Yichen;Zhang, Xiaoxi;Liang, Shu-Che;Joe-Wong, Carlee
• 通讯作者: Joe-Wong, Carlee

Coded Edge Computing

• DOI: 10.1109/infocom41043.2020.9155226
• 发表时间: 2020-07
• 期刊: IEEE INFOCOM 2020 - IEEE Conference on Computer Communications
• 作者: Kwang Taik Kim;Carlee Joe-Wong;M. Chiang

Machine Learning on Volatile Instances

• DOI: 10.1109/infocom41043.2020.9155448
• 发表时间: 2020-03
• 期刊: IEEE INFOCOM 2020 - IEEE Conference on Computer Communications
• 作者: Xiaoxi Zhang;Jianyu Wang;Gauri Joshi;Carlee Joe-Wong

GCN-SE: Attention as Explainability for Node Classification in Dynamic Graphs

• DOI: 10.1109/icdm51629.2021.00123
• 发表时间: 2021-10
• 期刊: 2021 IEEE International Conference on Data Mining (ICDM)
• 作者: Yucai Fan;Yuhang Yao;Carlee Joe-Wong

Interpretable Clustering on Dynamic Graphs with Recurrent Graph Neural Networks

• DOI: 10.1609/aaai.v35i5.16590
• 发表时间: 2020-12
• 作者: Yuhang Yao;Carlee Joe-Wong