SHF: Medium: Exploring an Edge Platform Design Trajectory for Next Generation XR Applications

延展实境（XR）包括虚拟现实（VR）和增强现实（AR），其中，在频谱的一端，用户可以探索计算机生成的环境并与之交互，在频谱的另一端，用户的物理世界的一部分通过计算机生成的数据增强，这是一种新兴技术，在许多国家重要的领域具有变革潜力，包括科学、医疗保健、教育、娱乐和商业。虽然目前市面上有各种类型的XR产品（有线、独立、无屏幕），但它们只为成本、性能和功耗不同的非常特定的应用提供点解决方案。此外，由于许多XR应用程序将在边缘计算场景中运行，因此其设计不仅需要考虑功率和能源因素，还需要适应性和现场定制，以有效支持任何未来的应用程序。因此，迫切需要开发框架，使设计人员能够生成，测试，评估和定制大量的下一代XR设备，并提供相应的软件支持。这些框架可以加快基于AR/VR的应用程序的发现时间，并重新定义计算的未来。该项目旨在设计一个灵活且具有成本效益的硬件和软件平台AVATAR，该平台可以根据各种XR应用程序的性能，服务质量（QoS）和能效需求进行逐步调整和定制。 AVATAR由四个中间的推进器组成。首先，对三种不同边缘平台上的四种XR应用进行详细分析，以了解XR管道中不同阶段的延迟，功率和利用率，以识别瓶颈阶段/内核。其次，它承担了编译器和运行时系统的协调设计，以控制可调旋钮，如代码分区和动态重配置。接下来，硬件设计替代方案，包括设备异构性，如通用CPU内核，图形处理单元（GPU），和现场可编程门阵列（FPGA）的XR应用程序的目标内核/流水线阶段，沿着可重新配置的范围，探索所需的性能-功率权衡。此外，还研究了利用Wi-Fi/5G通信来促进边缘设备和云之间的有效计算划分的范围。最后，正在开发一个综合评估框架，以评估针对不同XR应用提出的解决方案的有效性。 在教育方面，AVATAR包括一个新的专题课程，本科生和研究生参与这个新兴的研究领域，学生可以接触到计算机体系结构，编译器，运行时系统和网络中的交叉主题，以及XR应用程序的新类别。它还包括几个扩大参与计算（BPC）活动，如针对女孩的夏令营和与宾夕法尼亚州立大学教育部的合作，使K-12学生接触计算机科学和工程的许多领域，主要关注XR应用。该奖项反映了NSF的法定使命，并通过利用基金会的智力价值和知识产权进行评估，被认为值得支持更广泛的影响审查标准。

Extended Reality (XR) encompassing virtual reality (VR), where a computer-generated environment can be explored and interacted with by users, at one end of the spectrum and augmented reality (AR), where parts of users' physical world are enhanced with computer-generated data, at the other end of the spectrum, is an emerging technology with transformational potential in many domains of national importance including science, healthcare, education, entertainment, and commerce. While various flavors of XR products (tethered, standalone, screenless) are commercially available today, they only offer point solutions for very specific applications varying in cost, performance and power envelope. Furthermore, as many XR applications will run in an edge-computing scenario, their design demands not only power and energy considerations, but also the adaptability and field-customization to efficiently support any future applications. As a result, there is a critical need to develop frameworks that enable designers to generate, test, evaluate and customize a large spectrum of next-generation XR devices with accompanying software support. Such frameworks can expedite time-to-discovery for AR/VR-based applications and redefine the future of computing. This project seeks to design an agile and cost-effective hardware and software platform, AVATAR, that can be progressively adapted and customized to the performance, quality-of-service (QoS), and energy-efficiency needs of a diverse set of XR applications. AVATAR consists of four inter-twined thrusts. First, a detailed profiling of four types of XR applications on three different edge platforms is conducted to understand the latency, power and utilization of different stages in an XR pipeline for identifying the bottleneck stages/kernels. Second, it undertakes a coordinated design of the compiler and runtime system for controlling tunable knobs such as code partitioning and dynamic reconfiguration. Next, hardware design alternatives embracing device heterogeneity such as general-purpose CPU cores, graphic processing units (GPUs), and field programmable gate arrays (FPGAs) for the targeted kernels/pipeline stages of an XR application along with scope for reconfigurability are explored for desired performance-power tradeoffs. In addition, the scope for exploiting Wi-Fi/5G communication to facilitate efficient computation partitioning between an edge device and cloud is investigated. Finally, a comprehensive evaluation framework to evaluate the efficacy of the proposed solutions for different XR applications is being developed. On the educational front, AVATAR includes a new special-topics course and involvement of undergraduate and graduate students in this emerging research field, where students get exposure to cross-cutting topics in computer architecture, compilers, runtime systems and networking, as well as new classes of XR applications. It also includes several Broadening Participation in Computing (BPC) activities such as a summer camp targeting girls and collaboration with the Education Department at Penn State to expose K-12 students to many areas of computer science and engineering, with the primary focus being on XR applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.