FoMR: Shrinking the Control and Data Flow Latencies of Single Thread Executions for Emerging Workloads

AR/VR眼镜、智能相机、游戏机等边缘设备正在迅速渗透到社会中，用于执行监控、多用户游戏、沉浸式社交网络、自主控制等众多应用。虽然人们希望在这些边缘设备上执行所有数据处理，但由于资源限制，需要访问更广泛的数据库，因此需要将计算卸载到后端服务器。这些计算通常是并行的，以满足其吞吐量需求，但最慢的线程仍然会影响边缘设备/用户的延迟/响应。 因此，这些工作负载的高度交互性使得必须确保没有线程被留下，并且每个线程的单独数据路径被最大限度地利用。该项目旨在解决这一挑战，并努力消除在边缘设备上运行的各种应用程序之间的响应不匹配。研究人员的目标是提供一个研究生水平的课程在这个问题上，帮助提高相关的本科课程，举办夏令营的中学女生。这个项目提出了三个优化，每个减少控制和数据流延迟在正常的数据路径的新兴边缘服务工作负载。三个控制流优化包括：（i）用于更好的指令局部性的跨栈代码布局，（ii）识别和提升关键指令链（CritIC）的获取带宽，以及（iii）通过利用应用/数据知识来提高分支预测准确性，尤其是在共享库/OS中。在数据流方面，这三种优化尝试利用数据内容的重要性：（i）识别跨整个软件栈的频繁数据流序列并记忆它们，（ii）开发用于代码生成的内容感知的平铺技术以减少相同/相似内容之间的重用距离，以及（iii）该奖项反映了NSF的法定使命，并通过使用基金会的智力价值进行评估，更广泛的影响审查标准。

Edge devices such as AR/VR glasses, smart cameras, gaming consoles are permeating rapidly through society for performing numerous applications in surveillance, multi-user gaming, immersive social networking, autonomous control, etc. While one would want to perform all of the data processing at these edge devices, resource constraints with the need for accessing a broader corpus of data, require offloading the computations to a back-end server. These computations are usually parallelized for their throughput needs, but the slowest thread still impacts the latency/responsiveness to the edge device/user. The highly interactive nature of these workloads, thus, makes it imperative to ensure that no thread is left behind, and individual data path of each thread is maximally utilized. This project seeks to address this challenge and work to eliminate the mismatch in responsiveness across various applications run at edge devices. The investigators aim to offer a graduate level course on this subject, help enhance the related undergraduate courses, conduct summer camps for middle school girls.This project proposes three optimizations each for reducing control and data flow latencies in the normal data path for the emerging edge-serving workloads. The three control flow optimizations include: (i) cross-stack code layouts for better instruction locality, (ii) identifying and boosting the fetch bandwidth of critical instruction chains (CritICs), and (iii) improving branch prediction accuracy especially in shared libraries/OS by leveraging application/data knowledge. On the data flow side, the three optimizations try to leverage the importance of data content: (i) identifying frequent data flow sequences across the entire software stack and memorizing them, (ii) developing content-aware tiling techniques for code generation to reduce re-use distance between same/similar content, and (iii) dynamically leveraging approximations.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.