SHF: Medium: Collaborative Research: Enhancing Mobile VR/AR User Experience: An Integrated Architecture-System Approach

虚拟现实（VR）和增强现实（AR）正在经历快速增长，并在计算硬件性能显著提高和图形和显示技术革命的推动下进入主流市场。智能手机越来越多地采用VR和AR技术，为移动用户提供丰富的体验。然而，移动设备有限的硬件资源很难支撑VR/AR技术对资源的巨大需求和对良好用户体验的需求。在智能手机上启用VR/AR应用程序时，有几个关键挑战会显著降低用户体验。例如，当VR用户的眼睛轻微移动时，计算机无法及时提供与新视图相对应的图像。这种延迟导致运动异常和不满意的用户体验。移动VR/AR面临的其他挑战是有限的电池寿命和增加的散热。该项目的研究人员正在利用VR/AR的独特功能，探索协同架构系统方案，以解决上述性能，电池寿命和热挑战，从而增强移动VR/AR用户体验。该项目将为下一代移动平台打开大门，为移动用户提供高质量的低功耗VR/AR服务。该项目还将通过参与代表性不足的群体，向高中生推广，开发物联网课程，传播教育和培训研究基础设施，为社会做出贡献。该项目包含四个目标，包括：(1)近似计算以消除移动AR/VR中不必要的工作负载，在不牺牲用户感知的图像质量和输出精度的情况下实现性能，功耗和散热的三重优势；(2)新兴技术（如内存处理、多芯片模块(MCM)-GPU）支持移动VR/AR的性能/功耗优化；(3)利用动态热能收集来冷却热点，延长电池寿命，提高性能；(4)整合关键研究创新和交叉技术优化，最大限度地提高性能/功率/热性能。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Virtual reality (VR) and augmented reality (AR) are now experiencing rapid growth and debuting into mainstream markets driven by the significant performance improvement of computing hardware and the revolution of graphics and display technologies. Both VR and AR are increasingly adopted in smartphones for providing rich experience to mobile users. However, the limited hardware resources in mobile devices can barely support the tremendous resource requirements by VR/AR technology and the demand for a good user experience. There are several key challenges that dramatically degrade user experience when enabling VR/AR apps on smartphones. For example, when the VR user shifts her/his eyes slightly, the computer is unable to provide an image that corresponds to the new view in a timely manner. This delay causes motion anomalies and unsatisfying user experience. Other challenges for mobile VR/AR are limited battery life and increased heat dissipation. The investigators on this project are leveraging the unique features of VR/AR and exploring a synergetic architecture-system program to tackle the above performance, battery life, and thermal challenges, thus, enhancing mobile VR/AR user experience. This project will open the door for next generation mobile platforms that provide high-quality low-power VR/AR services to satisfy mobile users. This project will also contribute to society through engaging under-represented groups, and outreach to high-school students, curriculum development on Internet of Things, and disseminating research infrastructure for education and training.This project contains four objectives, including (1) approximate computing to eliminate unnecessary workloads in mobile AR/VR to achieve threefold benefits on performance, power, and thermal without sacrificing the user perceived image quality and output accuracy; (2) emerging technologies (e.g., processing-in-memory, multiple chip module (MCM)-GPU) enabled mobile VR/AR for performance/power optimization; (3) exploiting dynamic thermal energy harvesting to cool hotspots, prolong battery life and improve performance; and (4) integration of the key research innovations and cross-technology optimizations to maximize the performance/power/thermal enhancement.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.