SHF: Medium: Exploring an Edge Platform Design Trajectory for Next Generation XR Applications

扩展现实(XR)包括虚拟现实(VR)，其中用户可以探索和交互计算机生成的环境，在频谱的一端，增强现实(AR)是一种新兴技术，在许多具有国家重要性的领域，包括科学、医疗保健、教育、娱乐和商业，增强现实(AR)通过计算机生成的数据增强用户的物理世界。虽然各种风格的XR产品(系留、独立、无屏幕)今天都可以在商业上买到，但它们只为成本、性能和功率包络不同的特定应用提供点式解决方案。此外，由于许多XR应用程序将运行在边缘计算场景中，它们的设计不仅需要考虑功率和能源，还需要适应性和现场定制，以有效地支持任何未来的应用程序。因此，迫切需要开发框架，使设计人员能够生成、测试、评估和定制大量的下一代XR设备以及附带的软件支持。这样的框架可以加快基于AR/VR的应用程序的发现时间，并重新定义计算的未来。该项目旨在设计一个灵活且经济实惠的硬件和软件平台--阿凡达，该平台可以逐步调整和定制，以适应各种XR应用程序的性能、服务质量(Qos)和能效需求。《阿凡达》由四个相互交织的推力组成。首先，在三个不同的边缘平台上对四种类型的XR应用进行了详细的剖析，以了解XR流水线中不同阶段的延迟、功耗和利用率，以识别瓶颈阶段/内核。其次，它负责编译器和运行时系统的协调设计，以控制可调旋钮，如代码分区和动态重新配置。接下来，探讨了包含设备异构性的硬件设计备选方案，例如用于XR应用的目标内核/流水线阶段的通用CPU核心、图形处理单元(GPU)和现场可编程门阵列(FGA)，以及可重新配置的范围，以实现所需的性能-功率折衷。此外，还研究了利用Wi-Fi/5G通信来促进边缘设备和云之间的高效计算分区的范围。最后，正在制定一个全面的评价框架，以评估所提议的解决方案对不同XR应用的效果。在教育方面，阿凡达开设了一门新的专题课程，让本科生和研究生参与到这个新兴的研究领域中来，在这个领域，学生们可以接触到计算机架构、编译器、运行时系统和网络等交叉主题，以及XR应用程序的新课程。它还包括几项扩大计算参与度(BPC)的活动，如面向女孩的夏令营，以及与宾夕法尼亚州立大学教育部合作，让K-12学生接触到计算机科学和工程的许多领域，主要重点是XR应用。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Extended Reality (XR) encompassing virtual reality (VR), where a computer-generated environment can be explored and interacted with by users, at one end of the spectrum and augmented reality (AR), where parts of users' physical world are enhanced with computer-generated data, at the other end of the spectrum, is an emerging technology with transformational potential in many domains of national importance including science, healthcare, education, entertainment, and commerce. While various flavors of XR products (tethered, standalone, screenless) are commercially available today, they only offer point solutions for very specific applications varying in cost, performance and power envelope. Furthermore, as many XR applications will run in an edge-computing scenario, their design demands not only power and energy considerations, but also the adaptability and field-customization to efficiently support any future applications. As a result, there is a critical need to develop frameworks that enable designers to generate, test, evaluate and customize a large spectrum of next-generation XR devices with accompanying software support. Such frameworks can expedite time-to-discovery for AR/VR-based applications and redefine the future of computing. This project seeks to design an agile and cost-effective hardware and software platform, AVATAR, that can be progressively adapted and customized to the performance, quality-of-service (QoS), and energy-efficiency needs of a diverse set of XR applications. AVATAR consists of four inter-twined thrusts. First, a detailed profiling of four types of XR applications on three different edge platforms is conducted to understand the latency, power and utilization of different stages in an XR pipeline for identifying the bottleneck stages/kernels. Second, it undertakes a coordinated design of the compiler and runtime system for controlling tunable knobs such as code partitioning and dynamic reconfiguration. Next, hardware design alternatives embracing device heterogeneity such as general-purpose CPU cores, graphic processing units (GPUs), and field programmable gate arrays (FPGAs) for the targeted kernels/pipeline stages of an XR application along with scope for reconfigurability are explored for desired performance-power tradeoffs. In addition, the scope for exploiting Wi-Fi/5G communication to facilitate efficient computation partitioning between an edge device and cloud is investigated. Finally, a comprehensive evaluation framework to evaluate the efficacy of the proposed solutions for different XR applications is being developed. On the educational front, AVATAR includes a new special-topics course and involvement of undergraduate and graduate students in this emerging research field, where students get exposure to cross-cutting topics in computer architecture, compilers, runtime systems and networking, as well as new classes of XR applications. It also includes several Broadening Participation in Computing (BPC) activities such as a summer camp targeting girls and collaboration with the Education Department at Penn State to expose K-12 students to many areas of computer science and engineering, with the primary focus being on XR applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.