FoMR: Speculative Super-optimization: Boosting Performance via Speculation-Driven Dynamic Binary Optimization

现代处理器的特点是增加了单个多核处理器上的“核”计数，其中“核”是一个计算单元，允许每个核与处理器上的其他核并行运行，并且独立于处理器上的其他核运行。然而，相当大一部分软件应用程序本质上是连续的。尽管现代编译器具有复杂的优化功能，但由于在编译时不可预测的计算模式，这些多核之间的计算资源仍然会出现显著的浪费。该项目探索了在微处理器中部署积极的推测性动态优化的技术，从而实现对固有顺序代码的持续优化。这项工作解决了对系统的迫切需求，该系统能够积极且无缝地超级优化机器代码，以适应动态执行环境，从而加速计算机上的应用程序的执行。这项研究还将促进研究人员现有的努力和倡议，以建立一个来自不同背景的学生的管道。该项目探索处理器级别的动态二进制优化技术，通过利用已建立的推测性处理器功能，如分支预测、值预测和循环流检测，推测地生成和执行超级优化的指令流。该项目引入了两种截然不同的推测性超级优化：(A)利用动态预测的程序状态对短指令序列执行简单但功能强大的窥视孔优化的硬件实现，以及(B)利用作为辅助线程运行的基于微码的动态重新编译器来执行更复杂的优化的固件实现。由于有过多的推测性处理器功能和编译器/运行时/硬件优化可供选择，该项目探索了大量推测性超级优化策略的丰富样本空间，以及广泛的盈利分析，以动态确定适当的超级优化目标。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Modern processors are characterized by increasing 'core' counts on a single multi-core processor, where a 'core' is a computing unit, allowing each core to operate in parallel to, and independently of, other cores on the processor. Yet, a substantial chunk of software applications is inherently sequential. Although modern compilers feature sophisticated optimizations, significant waste of computational resources across these multiple-cores still occurs due to computational patterns that are unpredictable at compile-time. This project explores techniques to deploy aggressive speculative dynamic optimizations within the micro-processor, enabling continuous optimization of inherently sequential code. This work addresses a pressing need for systems that can aggressively and yet seamlessly super-optimize machine code, adapting to the dynamic execution environment, and thereby speed up execution of applications on computers. This research will also foster existing efforts and initiatives of the investigators to build a pipeline of students from diverse backgrounds.This project explores dynamic binary optimization techniques at the processor level to speculatively generate and execute a super-optimized instruction stream, by leveraging established speculative processor features such as branch prediction, value prediction, and loop stream detection. This project introduces two distinct flavors of speculative super-optimization: (a) a hardware implementation that leverages dynamically predicted program state to perform simple, yet powerful peephole optimizations on short instruction sequences, and (b) a firmware implementation that leverages a microcode-based dynamic re-compiler running as a helper thread to perform more sophisticated optimizations. Owing to the plethora of speculative processor features and compiler/runtime/hardware optimizations to choose from, this project explores a rich sample space of speculative super-optimization strategies, along with extensive profitability analysis to dynamically identify appropriate targets for super-optimization.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.