FoMR: Shrinking the Control and Data Flow Latencies of Single Thread Executions for Emerging Workloads

AR/VR眼镜、智能相机、游戏机等边缘设备正在迅速渗透到社会中，用于执行监控、多用户游戏、沉浸式社交网络、自主控制等方面的大量应用。虽然人们希望在这些边缘设备上执行所有数据处理，但由于需要访问更广泛的数据语料库，资源限制需要将计算卸载到后端服务器。这些计算通常是并行的，以满足其吞吐量需求，但最慢的线程仍会影响对边缘设备/用户的延迟/响应。因此，这些工作负载的高度交互特性使得必须确保不落下任何线程，并最大限度地利用每个线程的单独数据路径。该项目旨在解决这一挑战，并致力于消除在边缘设备上运行的各种应用程序在响应方面的不匹配。研究人员的目标是在这个主题上提供研究生水平的课程，帮助加强相关的本科课程，为中学女生举办夏令营。该项目提出了三项优化，每项优化都是为了减少新兴边缘服务工作负载在正常数据路径上的控制和数据流延迟。这三个控制流优化包括：(I)跨堆栈代码布局以获得更好的指令局部性，(Ii)识别和提高关键指令链(批评者)的获取带宽，以及(Iii)通过利用应用程序/数据知识来提高分支预测精度，特别是在共享库/OS中。在数据流方面，这三个优化试图利用数据内容的重要性：(I)识别整个软件堆栈中的频繁数据流序列并将其记住，(Ii)开发内容感知的代码生成平铺技术，以减少相同/相似内容之间的重复使用距离，以及(Iii)动态利用近似。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Edge devices such as AR/VR glasses, smart cameras, gaming consoles are permeating rapidly through society for performing numerous applications in surveillance, multi-user gaming, immersive social networking, autonomous control, etc. While one would want to perform all of the data processing at these edge devices, resource constraints with the need for accessing a broader corpus of data, require offloading the computations to a back-end server. These computations are usually parallelized for their throughput needs, but the slowest thread still impacts the latency/responsiveness to the edge device/user. The highly interactive nature of these workloads, thus, makes it imperative to ensure that no thread is left behind, and individual data path of each thread is maximally utilized. This project seeks to address this challenge and work to eliminate the mismatch in responsiveness across various applications run at edge devices. The investigators aim to offer a graduate level course on this subject, help enhance the related undergraduate courses, conduct summer camps for middle school girls.This project proposes three optimizations each for reducing control and data flow latencies in the normal data path for the emerging edge-serving workloads. The three control flow optimizations include: (i) cross-stack code layouts for better instruction locality, (ii) identifying and boosting the fetch bandwidth of critical instruction chains (CritICs), and (iii) improving branch prediction accuracy especially in shared libraries/OS by leveraging application/data knowledge. On the data flow side, the three optimizations try to leverage the importance of data content: (i) identifying frequent data flow sequences across the entire software stack and memorizing them, (ii) developing content-aware tiling techniques for code generation to reduce re-use distance between same/similar content, and (iii) dynamically leveraging approximations.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.