Equality Saturation for Deep Learning Compilers

点击翻译按钮获取中文摘要

Exponential growth in the computational demands of state-of-the-art Deep Learning (DL) models poses both societal andenvironmental concerns as research is increasingly monopolized by corporate labs which can afford multi-million dollartraining processes which in turn produce tens of thousands of kilograms of CO2. Improving the efficiency of DL istherefore paramount to democratizing research and protecting the planet. Equality Saturation, a novel compileroptimization technique, is one promising approach, reducing DL runtime by up to 70% and finding such improvements300x faster than traditional methods in some scenarios. However, its widespread application to Deep Learning Compilersis limited by its complexity, scalability and the difficulty in adapting it to the heterogeneous DL frameworks availabletoday. By integrating Equality Saturation into the MLIR compiler, Equality Saturation could be widely applied acrossframeworks without requiring tens of thousands of lines of handwritten optimization code, thereby increasing itsaccessibility to developers and greatly improving the efficiency of modern DL systems.