SHF: Small: An Integrated Hardware-Software Architecture for Efficient, Low-Power, Spatially Collaborative Computing in Augmented Reality

新兴的移动的应用越来越关注与用户所占据的真实世界环境交互并增强用户所占据的真实世界环境的技术。增强现实是一种将虚拟对象放置在用户对真实的世界的视图上的技术，具有诸如导航、游戏和教育的广泛应用。增强现实作为一种技术，其计算量非常大，导致资源受限的智能手机出现延迟、准确性和能耗问题。基于图像识别的增强现实通过需要整个图像识别流水线的计算来解决这个问题。此外，移动的硬件在设计时并没有考虑到增强现实和基于图像的繁重计算。移动的缓存、多核、GPU和其他移动的架构没有充分发挥其潜力来帮助解决移动的增强现实所面临的问题。该项目探索了以新的方式利用现成智能手机的独特移动的架构的方法，以在各种各样的移动的设备上实现增强现实。具体来说，增强现实已经成为从K-12到大学和毕业后教育的各级教育工作者的重要工具。该项目将使这种新的教育技术在世界上得到更广泛的应用。该项目的目标是使智能手机能够通过同时高效无缝地计算图像识别和世界跟踪任务来支持增强现实，其中包括三个研究部分。(1)通过近似跟踪研究基于智能手机的增强现实的基本问题，以减少计算和能量负载的方式提供高质量的对象跟踪。(2)研究软件定义的缓存技术，利用移动的增强现实的独特性质，提供专为高度协作的设备到设备增强现实设计的缓存。(3)该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Emerging mobile applications are increasingly focused on technology interacting with and augmenting the real-world environment the user occupies. Augmented reality is a technology that places virtual objects on a user’s view of the real world with a wide range of applications such as navigation, gaming, and education. Augmented reality as a technology is inherently extremely computation-heavy, leading to latency, accuracy, and energy-consumption issues on resource-constrained smartphones. Image-recognitio- based augmented reality compounds this issue by requiring the computation of the entire image-recognition pipeline. In addition, mobile hardware is not designed with augmented reality and heavy image-based computations in mind. Mobile caching, multicores, GPUs and other mobile architectures are not being utilized to their full potential to help resolve the issues plaguing mobile augmented reality. This project explores methods to utilize the unique mobile architecture of off-the-shelf smartphones in new ways to realize augmented reality on a wide variety of mobile devices. Specifically, augmented reality has become an important tool for educators at all levels from K-12 all the way through collegiate and post-graduation education. This project will allow for this new educational technology to be more widely utilized in the world.The objective of this project is to enable smartphones to support augmented reality via efficiently and seamlessly computing image-recognition and world-tracking tasks simultaneously, with three research components. (1) Investigate the foundational issues of smartphone-based augmented-reality through approximate-tracking to provide high-quality object tracking at reduced computational and energy loads. (2) Research software-defined caching techniques to utilize the unique nature of mobile augmented reality to provide caching specially designed for highly collaborative device-to-device augmented reality. (3) Explore hardware-based techniques relating to GPGPU computation and cache management to facilitate fast and efficient image-recognition and augmented-reality tasks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.