SHF: Small: An Integrated Hardware-Software Architecture for Efficient, Low-Power, Spatially Collaborative Computing in Augmented Reality

新兴的移动应用程序越来越关注与用户所处的现实世界环境进行交互和增强的技术。增强现实是一种将虚拟物体置于用户对现实世界的看法上的技术，具有广泛的应用，如导航、游戏和教育。增强现实作为一种技术，其固有的计算量非常大，在资源有限的智能手机上导致延迟、准确性和能耗问题。基于图像识别的增强现实需要计算整个图像识别管道，从而使这个问题复杂化。此外，移动硬件在设计时并没有考虑到增强现实和大量基于图像的计算。移动缓存、多核、gpu和其他移动架构并没有充分发挥其潜力来帮助解决困扰移动增强现实的问题。该项目探索了如何利用现有智能手机的独特移动架构，以新的方式在各种移动设备上实现增强现实。具体来说，增强现实已经成为各级教育工作者的重要工具，从K-12一直到大学和毕业后的教育。这个项目将使这种新的教育技术在世界上得到更广泛的应用。该项目的目标是通过同时高效无缝地计算图像识别和世界跟踪任务，使智能手机能够支持增强现实，包括三个研究组成部分。(1)通过近似跟踪研究基于智能手机的增强现实的基本问题，在减少计算和能量负荷的情况下提供高质量的目标跟踪。(2)研究软件定义缓存技术，利用移动增强现实的独特性，为设备对设备的高度协作增强现实提供专门设计的缓存。(3)探索与GPGPU计算和缓存管理相关的基于硬件的技术，以促进快速高效的图像识别和增强现实任务。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Emerging mobile applications are increasingly focused on technology interacting with and augmenting the real-world environment the user occupies. Augmented reality is a technology that places virtual objects on a user’s view of the real world with a wide range of applications such as navigation, gaming, and education. Augmented reality as a technology is inherently extremely computation-heavy, leading to latency, accuracy, and energy-consumption issues on resource-constrained smartphones. Image-recognitio- based augmented reality compounds this issue by requiring the computation of the entire image-recognition pipeline. In addition, mobile hardware is not designed with augmented reality and heavy image-based computations in mind. Mobile caching, multicores, GPUs and other mobile architectures are not being utilized to their full potential to help resolve the issues plaguing mobile augmented reality. This project explores methods to utilize the unique mobile architecture of off-the-shelf smartphones in new ways to realize augmented reality on a wide variety of mobile devices. Specifically, augmented reality has become an important tool for educators at all levels from K-12 all the way through collegiate and post-graduation education. This project will allow for this new educational technology to be more widely utilized in the world.The objective of this project is to enable smartphones to support augmented reality via efficiently and seamlessly computing image-recognition and world-tracking tasks simultaneously, with three research components. (1) Investigate the foundational issues of smartphone-based augmented-reality through approximate-tracking to provide high-quality object tracking at reduced computational and energy loads. (2) Research software-defined caching techniques to utilize the unique nature of mobile augmented reality to provide caching specially designed for highly collaborative device-to-device augmented reality. (3) Explore hardware-based techniques relating to GPGPU computation and cache management to facilitate fast and efficient image-recognition and augmented-reality tasks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.