SpDISTAL: Compiling Distributed Sparse Tensor Computations
SpDISTAL: Compiling Distributed Sparse Tensor Computations
复制标题
DOI：
10.1109/sc41404.2022.00064
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Rohan Yadav;Alex Aiken;Fredrik Kjolstad
中科院分区：
文献类型：
--
作者：
Rohan Yadav;Alex Aiken;Fredrik Kjolstad
我们介绍了针对分布式系统的稀疏张量代数的编译器Spdistal。 Spdistal结合了张量代数表达式，稀疏数据结构，数据分布和计算分布的单独描述。因此，它可以通过各种稀疏的数据结构和数据分布来分布稀疏张量代数表达式的分布式执行。 SPDistal被实现为C ++库，该库针对基于任务的运行时系统，并可以为具有多核CPU和多个GPU的节点生成代码。 SPDISTAL生成分布式代码，该代码可通过手工编写的分布式功能来实现特定稀疏张量代数表达式的竞争性竞争，并以一到两个数量级优于基于一般解释的系统。
We introduce SpDISTAL, a compiler for sparse tensor algebra that targets distributed systems. SpDISTAL combines separate descriptions of tensor algebra expressions, sparse data structures, data distribution, and computation distribution. Thus, it enables distributed execution of sparse tensor algebra expressions with a wide variety of sparse data structures and data distributions. SpDISTAL is implemented as a C++ library that targets a distributed task-based runtime system and can generate code for nodes with both multi-core CPUs and multiple GPUs. SpDISTAL generates distributed code that achieves performance competitive with hand-written distributed functions for specific sparse tensor algebra expressions and that outperforms general interpretation-based systems by one to two orders of magnitude.