CAREER: Enhancing Energy Efficiency in Mobile Augmented Reality Apps

移动的增强现实（MAR）应用利用移动的设备的摄像头来利用一组虚拟对象来增强真实的用户环境。因此，用户可以通过经由移动的用户界面与增强环境交互来实现高水平的参与。典型的MAR应用程序需要执行多个计算密集型任务来渲染虚拟对象，并使用人工智能（AI）分析环境。然而，由于其相对小的形状因子，移动的设备仅具有有限的计算资源和能量可用于MAR应用执行，这可能导致电池寿命短和用户体验差。因此，为了确保MAR未来的成功，不仅要对如何在MAR应用程序中实现高能效进行新的研究，而且还要对未来的员工进行节能移动的软件设计的最佳实践教育。为了实现这些目标，该项目将为MAR应用程序提出一个边缘辅助和节能的框架。具体而言，该项目设计了运行时算法，（1）在每个移动终端上，通过操纵虚拟对象质量，AI模型复杂度和异构计算资源上的AI任务分配来控制用户感知的性能和能耗，（2）在支持边缘服务器上，通过对相似的虚拟对象抽取请求进行分组以最小化交叉，在有限的边缘计算和存储资源下的用户性能成本，以及（3）将在物理测试平台上实施和测试。通过提供使MAR应用程序更节能的框架，该项目将有助于加快增强现实在现代社会许多不同关键领域的采用，如教育和劳动力培训，并将释放新技术的全部潜力，例如增强远程会议和公共场所的增强体验。该项目还将创建新的研究生课程，以教育未来的劳动力进行节能移动的资源管理，为本科生提供新的研究机会，并为K12学生提供新的夏令营。课程材料将提供给有兴趣引进类似课程、项目和夏令营的其他大学。该项目的成果将通过在顶级会议和期刊上发表的出版物传播。同样，所有提出的算法的源代码将作为开放源代码提供。该奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

Mobile Augmented Reality (MAR) apps leverage the camera of mobile devices to augment the real user environment with a set of virtual objects. Users can thus achieve a high level of engagement by interacting with the augmented environment through the mobile user interface. Typical MAR apps require executing several compute-intensive tasks to render virtual objects and, using Artificial Intelligence (AI), analyze the environment. However, due to their relatively small form factor, mobile devices only have limited computing resources and energy available for the MAR app execution, which may lead to short battery life and poor user experience. To ensure the future success of MAR, it is thus important not only to conduct novel research on how to enable high energy efficiency in MAR apps but also to educate the future workforce in the best practices for energy efficient mobile software design. To achieve these goals, this project will propose an edge-assisted and energy-efficient framework for MAR apps. Specifically, the project designs runtime algorithms that (1), on each mobile device, control user-perceived performance and energy consumption by manipulating virtual object quality, AI-model complexity, and AI-task allocation on the heterogeneous computing resources, (2), on the supporting edge servers, ensure scalability by grouping similar virtual object decimation requests for minimized cross-user performance cost under limited edge computing and storage resources, and (3) will be implemented and tested on a physical testbed.By providing a framework that makes MAR apps more energy-efficient, this project will help speed up the adoption of augmented reality in many different key sectors of modern society, such as education and workforce training, and will unlock the full potential of new technologies such as augmented remote meetings and augmented experiences in public places. The project will also result in the creation of new graduate-level courses to educate the future workforce on energy-efficient mobile resource management, new research opportunities for undergraduate students, and new summer camps for K12 students. The course material will be made available to other universities interested to introduce similar courses, projects, and summer camps. The results of this project will be disseminated through publications in top-tier conferences and journals. Similarly, all the source code of the proposed algorithms will be made available as open source.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.