Challenges in Migrating Imperative Deep Learning Programs to Graph Execution: An Empirical Study
Challenges in Migrating Imperative Deep Learning Programs to Graph Execution: An Empirical Study
复制标题
DOI：
10.1145/3524842.3528455
复制
发表时间：
2022-01
期刊：
影响因子：
--
通讯作者：
Tatiana Castro V'elez;Raffi Khatchadourian;M. Bagherzadeh;A. Raja
中科院分区：
文献类型：
--
作者：
Tatiana Castro V'elez;Raffi Khatchadourian;M. Bagherzadeh;A. Raja
效率对于支持w.r.t.的响应至关重要。不断增长的数据集，特别是对于深度学习（DL）系统。DL框架传统上采用延迟执行风格的DL代码，支持符号化的、基于图形的深度神经网络（DNN）计算。虽然可扩展，但这样的开发往往会产生容易出错、不直观且难以调试的DL代码。因此，以牺牲运行时性能为代价，出现了更自然、更不易出错的命令式DL框架，鼓励急切执行。虽然混合方法的目标是“两全其美”，但在真实的世界中应用它们的挑战在很大程度上是未知的。我们进行了数据驱动的挑战和由此产生的错误，涉及编写可靠的，但性能的命令式DL代码的分析，通过研究250个开源项目，包括19.7 MPEG4，沿着与470和446手动检查的代码补丁和错误报告，分别。结果表明，杂交：（i）是容易API误用，（ii）可能会导致性能下降，其意图的相反，和（iii）由于执行模式不兼容的应用程序有限。我们提出了几个建议、最佳实践和反模式，以有效地混合命令式深度学习代码，这可能会使深度学习从业者、API设计者、工具开发人员和教育工作者受益。
Efficiency is essential to support responsiveness w.r.t. ever-growing datasets, especially for Deep Learning (DL) systems. DL frameworks have traditionally embraced deferred execution-style DL code that supports symbolic, graph-based Deep Neural Network (DNN) computation. While scalable, such development tends to produce DL code that is error-prone, non-intuitive, and difficult to debug. Consequently, more natural, less error-prone imperative DL frameworks encouraging eager execution have emerged at the expense of run-time performance. While hybrid approaches aim for the “best of both worlds,” the challenges in applying them in the real world are largely unknown. We conduct a data-driven analysis of challenges-and resultant bugs-involved in writing reliable yet performant imperative DL code by studying 250 open-source projects, consisting of 19.7 MLOC, along with 470 and 446 manually examined code patches and bug reports, respectively. The results indicate that hybridization: (i) is prone to API misuse, (ii) can result in performance degradation-the opposite of its intention, and (iii) has limited application due to execution mode incompatibility. We put forth several recommendations, best practices, and anti-patterns for effectively hybridizing imperative DL code, potentially benefiting DL practitioners, API designers, tool developers, and educators.