Distributed Halide
Distributed Halide
复制标题
DOI：
10.1145/2851141.2851157
复制
发表时间：
2016-02
期刊：
影响因子：
--
通讯作者：
Tyler Denniston;Shoaib Kamil;Saman P. Amarasinghe
中科院分区：
文献类型：
--
作者：
Tyler Denniston;Shoaib Kamil;Saman P. Amarasinghe
许多图像处理任务自然地表示为称为模板的小型计算核的管道。Halide是一种流行的领域特定语言和编译器，旨在实现图像处理算法。Halide使用简单的语言结构来表示要计算什么，并使用单独的调度协同语言来表示何时何地执行计算。这种方法已经证明了与手工优化代码相当或更好的性能。然而，到目前为止，Halide仅限于并行共享内存执行，这限制了它在内存带宽受限的管道或大规模图像处理任务中的性能。我们为Halide提供了一个扩展，以支持复杂模板管道的分布式内存并行执行。这些扩展与Halide中现有的调度结构组合在一起，允许表达复杂的计算和通信策略。现有的Halide应用程序可以通过最小的更改进行分发，这使得程序员可以轻松地在重新计算和通信之间进行权衡。即使是200行99阶段的应用程序，也需要大约10行新代码。在9个图像处理基准测试中，我们的扩展通过减轻非均匀内存访问的影响，在单个节点上比具有相同核数的常规多线程执行提供了1.4倍的加速。分布式基准测试在16节点的测试机上实现了高达18倍的加速，在NERSC Cori超级计算机的64节点上实现了高达57倍的加速。
Many image processing tasks are naturally expressed as a pipeline of small computational kernels known as stencils. Halide is a popular domain-specific language and compiler designed to implement image processing algorithms. Halide uses simple language constructs to express what to compute and a separate scheduling co-language for expressing when and where to perform the computation. This approach has demonstrated performance comparable to or better than hand-optimized code. Until now, however, Halide has been restricted to parallel shared memory execution, limiting its performance for memory-bandwidth-bound pipelines or large-scale image processing tasks. We present an extension to Halide to support distributed-memory parallel execution of complex stencil pipelines. These extensions compose with the existing scheduling constructs in Halide, allowing expression of complex computation and communication strategies. Existing Halide applications can be distributed with minimal changes, allowing programmers to explore the tradeoff between recomputation and communication with little effort. Approximately 10 new of lines code are needed even for a 200 line, 99 stage application. On nine image processing benchmarks, our extensions give up to a 1.4× speedup on a single node over regular multithreaded execution with the same number of cores, by mitigating the effects of non-uniform memory access. The distributed benchmarks achieve up to 18× speedup on a 16 node testing machine and up to 57× speedup on 64 nodes of the NERSC Cori supercomputer.