DyCL: Dynamic Neural Network Compilation Via Program Rewriting and Graph Optimization
DyCL: Dynamic Neural Network Compilation Via Program Rewriting and Graph Optimization
复制标题
DOI：
10.1145/3597926.3598082
复制
发表时间：
2023-07
期刊：
影响因子：
--
通讯作者：
Simin Chen;Shiyi Wei;Cong Liu;Wei Yang
中科院分区：
文献类型：
--
作者：
Simin Chen;Shiyi Wei;Cong Liu;Wei Yang
深度学习（DL）编译器是一个重要的基础设施组件，可以在各种硬件平台（如移动设备和树莓派）上部署深度神经网络。深度学习编译器的主要功能是将用高级深度学习框架（如PyTorch和TensorFlow）编写的深度学习程序转换为可移植的可执行文件。然后，这些可执行文件可以被部署的主机程序灵活地执行。然而，现有的DL编译器依赖于跟踪机制，该机制包括向神经网络程序提供运行时输入并跟踪程序执行路径，以生成编译所需的计算图。不幸的是，这种机制在处理现代动态神经网络（DyNNs）时存在不足，因为动态神经网络根据输入具有不同的计算图。因此，传统的DL编译器很难准确地将dynn编译成可执行代码。为了解决这个限制，我们提出了DyCL，这是一种通用的方法，可以使任何现有的DL编译器成功地编译dynn。DyCL通过引入一种编译机制来解决dynn的动态特性，该机制在编译过程中重新分配原始DNN程序的控制和数据流。具体来说，DyCL开发了程序分析和程序转换技术，将动态神经网络转换为多个子神经网络。每个子神经网络都没有条件语句，并且是独立编译的。此外，DyCL还综合了一个主机模块，该模块对dynn的控制流进行建模，并促进了子神经网络的调用。我们的评估证明了DyCL的有效性，在编译所有动态神经网络时实现了100%的成功率。此外，DyCL生成的编译后的可执行文件表现出显著的性能改进，运行速度比在通用DL框架上执行的原始dynn快1.12到20.21倍。
The deep learning (DL) compiler serves as a vital infrastructure component to enable the deployment of deep neural networks on diverse hardware platforms such as mobile devices and Raspberry Pi. DL compiler’s primary function is to translate DNN programs written in high-level DL frameworks such as PyTorch and TensorFlow into portable executables. These executables can then be flexibly executed by the deployed host programs. However, existing DL compilers rely on a tracing mechanism, which involves feeding a runtime input to a neural network program and tracing the program execution paths to generate the computational graph necessary for compilation. Unfortunately, this mechanism falls short when dealing with modern dynamic neural networks (DyNNs) that possess varying computational graphs depending on the inputs. Consequently, conventional DL compilers struggle to accurately compile DyNNs into executable code. To address this limitation, we propose DyCL, a general approach that enables any existing DL compiler to successfully compile DyNNs. DyCL tackles the dynamic nature of DyNNs by introducing a compilation mechanism that redistributes the control and data flow of the original DNN programs during the compilation process. Specifically, DyCL develops program analysis and program transformation techniques to convert a dynamic neural network into multiple sub-neural networks. Each sub-neural network is devoid of conditional statements and is compiled independently. Furthermore, DyCL synthesizes a host module that models the control flow of the DyNNs and facilitates the invocation of the sub-neural networks. Our evaluation demonstrates the effectiveness of DyCL, achieving a 100% success rate in compiling all dynamic neural networks. Moreover, the compiled executables generated by DyCL exhibit significantly improved performance, running between 1.12× and 20.21× faster than the original DyNNs executed on general-purpose DL frameworks.