SHF: Medium: Exploring an Edge Platform Design Trajectory for Next Generation XR Applications

扩展现实（XR）包括虚拟现实（VR）和增强现实（AR），前者是用户可以探索计算机生成的环境并与之交互，后者是用户物理世界的一部分通过计算机生成的数据得到增强，后者是一种新兴技术，在科学、医疗保健、教育、娱乐和商业等许多具有国家重要性的领域具有变革潜力。虽然目前市面上有各种各样的XR产品（捆绑、独立、无屏幕），但它们只针对成本、性能和功耗范围不同的特定应用提供点解决方案。此外，由于许多XR应用程序将在边缘计算场景中运行，它们的设计不仅需要考虑功率和能源，还需要适应性和现场定制，以有效地支持任何未来的应用程序。因此，迫切需要开发框架，使设计人员能够生成、测试、评估和定制大量下一代XR设备，并附带软件支持。这样的框架可以加快基于AR/ vr的应用程序的发现时间，并重新定义计算的未来。该项目旨在设计一种灵活且具有成本效益的硬件和软件平台AVATAR，该平台可以逐步适应和定制各种XR应用的性能、服务质量（QoS）和能效需求。AVATAR由四个相互缠绕的推力组成。首先，对三种不同边缘平台上的四种类型的XR应用程序进行了详细的分析，以了解XR管道中不同阶段的延迟、功耗和利用率，从而识别瓶颈阶段/内核。其次，对编译器和运行时系统进行协调设计，以控制代码分区和动态重配置等可调参数。接下来，硬件设计替代方案包括设备异构，如通用CPU内核、图形处理单元（gpu）和现场可编程门阵列（fpga），用于XR应用程序的目标内核/管道阶段，以及可重构性范围，以实现所需的性能-功率权衡。此外，还研究了利用Wi-Fi/5G通信促进边缘设备和云之间高效计算划分的范围。最后，正在开发一个综合评估框架，以评估针对不同XR应用的拟议解决方案的有效性。在教育方面，AVATAR包括一个新的专题课程，本科生和研究生参与这个新兴的研究领域，学生可以接触到计算机体系结构、编译器、运行时系统和网络的交叉主题，以及XR应用的新课程。它还包括一些扩大参与计算（BPC）活动，如针对女孩的夏令营，以及与宾夕法尼亚州立大学教育部合作，让K-12学生接触计算机科学和工程的许多领域，主要关注XR应用。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Extended Reality (XR) encompassing virtual reality (VR), where a computer-generated environment can be explored and interacted with by users, at one end of the spectrum and augmented reality (AR), where parts of users' physical world are enhanced with computer-generated data, at the other end of the spectrum, is an emerging technology with transformational potential in many domains of national importance including science, healthcare, education, entertainment, and commerce. While various flavors of XR products (tethered, standalone, screenless) are commercially available today, they only offer point solutions for very specific applications varying in cost, performance and power envelope. Furthermore, as many XR applications will run in an edge-computing scenario, their design demands not only power and energy considerations, but also the adaptability and field-customization to efficiently support any future applications. As a result, there is a critical need to develop frameworks that enable designers to generate, test, evaluate and customize a large spectrum of next-generation XR devices with accompanying software support. Such frameworks can expedite time-to-discovery for AR/VR-based applications and redefine the future of computing. This project seeks to design an agile and cost-effective hardware and software platform, AVATAR, that can be progressively adapted and customized to the performance, quality-of-service (QoS), and energy-efficiency needs of a diverse set of XR applications. AVATAR consists of four inter-twined thrusts. First, a detailed profiling of four types of XR applications on three different edge platforms is conducted to understand the latency, power and utilization of different stages in an XR pipeline for identifying the bottleneck stages/kernels. Second, it undertakes a coordinated design of the compiler and runtime system for controlling tunable knobs such as code partitioning and dynamic reconfiguration. Next, hardware design alternatives embracing device heterogeneity such as general-purpose CPU cores, graphic processing units (GPUs), and field programmable gate arrays (FPGAs) for the targeted kernels/pipeline stages of an XR application along with scope for reconfigurability are explored for desired performance-power tradeoffs. In addition, the scope for exploiting Wi-Fi/5G communication to facilitate efficient computation partitioning between an edge device and cloud is investigated. Finally, a comprehensive evaluation framework to evaluate the efficacy of the proposed solutions for different XR applications is being developed. On the educational front, AVATAR includes a new special-topics course and involvement of undergraduate and graduate students in this emerging research field, where students get exposure to cross-cutting topics in computer architecture, compilers, runtime systems and networking, as well as new classes of XR applications. It also includes several Broadening Participation in Computing (BPC) activities such as a summer camp targeting girls and collaboration with the Education Department at Penn State to expose K-12 students to many areas of computer science and engineering, with the primary focus being on XR applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.