SHF: Medium: Collaborative Research: Enhancing Mobile VR/AR User Experience: An Integrated Architecture-System Approach

在计算硬件性能显着提高以及图形和显示技术革命的推动下，虚拟现实（VR）和增强现实（AR）目前正在经历快速增长并首次进入主流市场。 VR和AR越来越多地在智能手机中采用，为移动用户提供丰富的体验。然而，移动设备有限的硬件资源难以支撑VR/AR技术巨大的资源需求和良好的用户体验需求。在智能手机上启用 VR/AR 应用程序时，存在几个会极大降低用户体验的关键挑战。例如，当VR用户轻微移动眼睛时，计算机无法及时提供与新视图相对应的图像。这种延迟会导致运动异常和令人不满意的用户体验。移动 VR/AR 面临的其他挑战包括电池寿命有限和散热增加。该项目的研究人员正在利用 VR/AR 的独特功能，探索协同架构系统方案来解决上述性能、电池寿命和散热挑战，从而增强移动 VR/AR 用户体验。该项目将为下一代移动平台打开大门，提供高质量低功耗 VR/AR 服务以满足移动用户的需求。该项目还将通过吸引代表性不足的群体、向高中生推广、物联网课程开发以及传播教育和培训研究基础设施等方式为社会做出贡献。该项目包含四个目标，包括（1）近似计算，以消除移动 AR/VR 中不必要的工作负载，以在不牺牲用户感知图像质量和输出精度的情况下实现性能、功耗和散热方面的三重优势； (2) 新兴技术（例如内存处理、多芯片模块 (MCM)-GPU）使移动 VR/AR 能够实现性能/功耗优化； (3) 利用动态热能收集来冷却热点、延长电池寿命并提高性能； (4) 整合关键研究创新和跨技术优化，以最大限度地提高性能/功率/热性能。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Virtual reality (VR) and augmented reality (AR) are now experiencing rapid growth and debuting into mainstream markets driven by the significant performance improvement of computing hardware and the revolution of graphics and display technologies. Both VR and AR are increasingly adopted in smartphones for providing rich experience to mobile users. However, the limited hardware resources in mobile devices can barely support the tremendous resource requirements by VR/AR technology and the demand for a good user experience. There are several key challenges that dramatically degrade user experience when enabling VR/AR apps on smartphones. For example, when the VR user shifts her/his eyes slightly, the computer is unable to provide an image that corresponds to the new view in a timely manner. This delay causes motion anomalies and unsatisfying user experience. Other challenges for mobile VR/AR are limited battery life and increased heat dissipation. The investigators on this project are leveraging the unique features of VR/AR and exploring a synergetic architecture-system program to tackle the above performance, battery life, and thermal challenges, thus, enhancing mobile VR/AR user experience. This project will open the door for next generation mobile platforms that provide high-quality low-power VR/AR services to satisfy mobile users. This project will also contribute to society through engaging under-represented groups, and outreach to high-school students, curriculum development on Internet of Things, and disseminating research infrastructure for education and training.This project contains four objectives, including (1) approximate computing to eliminate unnecessary workloads in mobile AR/VR to achieve threefold benefits on performance, power, and thermal without sacrificing the user perceived image quality and output accuracy; (2) emerging technologies (e.g., processing-in-memory, multiple chip module (MCM)-GPU) enabled mobile VR/AR for performance/power optimization; (3) exploiting dynamic thermal energy harvesting to cool hotspots, prolong battery life and improve performance; and (4) integration of the key research innovations and cross-technology optimizations to maximize the performance/power/thermal enhancement.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.