Differentiable Programming for Image Processing and Deep Learning in Halide
Differentiable Programming for Image Processing and Deep Learning in Halide
复制标题
DOI：
10.1145/3197517.3201383
复制
发表时间：
2018-08-01
影响因子：
6.2
通讯作者：
Ragan-Kelley, Jonathan
中科院分区：
文献类型：
--
作者：
Li, Tzu-Mao;Gharbi, Michael;Ragan-Kelley, Jonathan
基于梯度的优化通过深度学习和非线性优化等技术在计算成像方面取得了巨大的进步。这些方法不仅需要简单的数学函数的梯度，而且需要编码图像和图形数据的复杂变换的通用程序的梯度。不幸的是，从业者传统上被限制在手工派生复杂计算的梯度，或者在深度学习框架中使用有限的粗粒度操作符组合程序。同时，对于大多数程序员来说，编写具有成像和深度学习所需的性能水平的程序是非常困难的。我们扩展了图像处理语言Halide，使其具有通用反向模式自动微分(AD)，并具有自动优化实现梯度计算的能力。这使得能够以高性能自动计算任意Halide程序的梯度，而程序员只需很少的工作。一个关键的挑战是构造梯度代码以保持并行性。我们定义了一个简单的算法来自动调度这些流水线，并展示了Halide现有的调度原语如何表达和扩展“检查点”的关键AD优化。使用这个新工具，我们展示了如何轻松定义新的神经网络层，这些层自动编译为高性能的GPU实现，以及如何从计算成像解决非线性逆问题。最后，我们展示了可微编程如何能够显著提高甚至传统的前馈图像处理算法的质量，模糊了经典方法和深度方法之间的区别。
Gradient-based optimization has enabled dramatic advances in computational imaging through techniques like deep learning and nonlinear optimization. These methods require gradients not just of simple mathematical functions, but of general programs which encode complex transformations of images and graphical data. Unfortunately, practitioners have traditionally been limited to either hand-deriving gradients of complex computations, or composing programs from a limited set of coarse-grained operators in deep learning frameworks. At the same time, writing programs with the level of performance needed for imaging and deep learning is prohibitively difficult for most programmers.We extend the image processing language Halide with general reverse-mode automatic differentiation (AD), and the ability to automatically optimize the implementation of gradient computations. This enables automatic computation of the gradients of arbitrary Halide programs, at high performance, with little programmer effort. A key challenge is to structure the gradient code to retain parallelism. We define a simple algorithm to automatically schedule these pipelines, and show how Halide's existing scheduling primitives can express and extend the key AD optimization of "checkpointing."Using this new tool, we show how to easily define new neural network layers which automatically compile to high-performance GPU implementations, and how to solve nonlinear inverse problems from computational imaging. Finally, we show how differentiable programming enables dramatically improving the quality of even traditional, feed-forward image processing algorithms, blurring the distinction between classical and deep methods.