Collage: Seamless Integration of Deep Learning Backends with Automatic Placement
Collage: Seamless Integration of Deep Learning Backends with Automatic Placement
复制标题
DOI：
10.1145/3559009.3569651
复制
发表时间：
2021-11
期刊：
影响因子：
--
通讯作者：
Byungsoo Jeon;Sunghyun Park;Peiyuan Liao;Sheng Xu;Tianqi Chen;Zhihao Jia
中科院分区：
文献类型：
--
作者：
Byungsoo Jeon;Sunghyun Park;Peiyuan Liao;Sheng Xu;Tianqi Chen;Zhihao Jia
对高效和高性能部署深度学习应用程序的强烈需求促使了丰富的深度学习生态系统的快速发展。为了跟上这种快速发展，现代深度学习框架必须有效地集成各种优化的张量代数库和运行时作为其后端，并使用这些后端生成尽可能快的可执行文件。然而，当前的深度学习框架需要大量的手工工作和专业知识来集成每个新的后端，同时未能释放其全部潜力。考虑到深度学习生态系统的快速发展性质，这种手动方法往往会减缓跨不同层的持续创新；它阻止了硬件供应商快速部署他们的尖端库，DL框架开发人员必须反复调整他们的手工编码规则以适应新版本的库，机器学习从业者需要等待新技术的集成并经常遇到不满意的性能。在本文中，我们提出了Collage，这是一个提供深度学习后端无缝集成的深度学习框架。Collage提供了一个富有表现力的后端注册接口，允许用户精确地指定各种后端功能。通过利用可用后端规范，Collage可以自动为给定的工作负载和执行环境搜索优化后端放置策略。我们的评估显示，在NVIDIA的RTX 2070 GPU， V100 GPU和Intel的Xeon 8259CL CPU上，Collage在每种硬件上的平均性能分别比现有最佳框架高出1.26倍，1.43倍和1.40倍。Collage已经开源并部署在Apache TVM中。
The strong demand for efficient and performant deployment of Deep Learning (DL) applications prompts the rapid development of a rich DL ecosystem. To keep up with this fast advancement, it is crucial for modern DL frameworks to efficiently integrate a variety of optimized tensor algebra libraries and runtimes as their backends and generate the fastest possible executable using these backends. However, current DL frameworks require significant manual effort and expertise to integrate every new backend while failing to unleash its full potential. Given the fast-evolving nature of the DL ecosystem, this manual approach often slows down continuous innovations across different layers; it prevents hardware vendors from the fast deployment of their cutting-edge libraries, DL framework developers must repeatedly adjust their hand-coded rules to accommodate new versions of libraries, and machine learning practitioners need to wait for the integration of new technologies and often encounter unsatisfactory performance. In this paper, we propose Collage, a DL framework that offers seamless integration of DL backends. Collage provides an expressive backend registration interface that allows users to precisely specify the capability of various backends. By leveraging the specifications of available backends, Collage automatically searches for an optimized backend placement strategy for a given workload and execution environment. Our evaluation shows that Collage outperforms the best existing framework for each hardware by 1.26×, 1.43×, 1.40× on average on NVIDIA's RTX 2070 GPU, V100 GPU, and Intel's Xeon 8259CL CPU, respectively. Collage has been open-sourced 1 and deployed in Apache TVM.