Cortex: A Compiler for Recursive Deep Learning Models
Cortex: A Compiler for Recursive Deep Learning Models
复制标题
DOI：
--
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Pratik Fegade;Tianqi Chen;Phillip B. Gibbons;T. Mowry
中科院分区：
文献类型：
--
作者：
Pratik Fegade;Tianqi Chen;Phillip B. Gibbons;T. Mowry
优化深度学习模型通常分两个步骤进行：(I)高级图优化，如核融合；(Ii)低层核优化，如供应商库中找到的那些。这种方法通常会显著提高性能，尤其是在递归深度学习模型的情况下。在本文中，我们提出了一种基于编译器的方法Cortex，它可以为递归模型生成高效的代码，从而实现低延迟推理。我们的编译器方法和对供应商库的低依赖使我们能够执行端到端优化，导致跨不同后端的推理延迟比过去的工作减少多达14倍。
Optimizing deep learning models is generally performed in two steps: (i) high-level graph optimizations such as kernel fusion and (ii) low level kernel optimizations such as those found in vendor libraries. This approach often leaves significant performance on the table, especially for the case of recursive deep learning models. In this paper, we present Cortex, a compiler-based approach to generate highly-efficient code for recursive models for low latency inference. Our compiler approach and low reliance on vendor libraries enables us to perform end-to-end optimizations, leading to up to 14X lower inference latencies over past work, across different backends.