AutoTM: Automatic Tensor Movement in Heterogeneous Memory Systems using Integer Linear Programming
AutoTM: Automatic Tensor Movement in Heterogeneous Memory Systems using Integer Linear Programming
复制标题
DOI：
10.1145/3373376.3378465
复制
发表时间：
2020-03
期刊：
影响因子：
--
通讯作者：
Mark Hildebrand;J. Khan;Sanjeev N. Trika;Jason Lowe-Power;V. Akella
中科院分区：
文献类型：
--
作者：
Mark Hildebrand;J. Khan;Sanjeev N. Trika;Jason Lowe-Power;V. Akella
内存容量是训练大规模神经网络的关键瓶颈。英特尔® 傲腾#8482；以 NVDIMM 形式提供的 DC PMM（持久内存模块）是一项颠覆性技术，与传统 SSD 相比，它的读取带宽显着提高，而每比特成本却比传统 DRAM 更低。在这项工作中，我们展示了如何利用这种新的内存技术来最大限度地减少所需的 DRAM 数量，而不会显着影响性能。具体来说，我们利用深度神经网络应用中底层计算图的静态特性，开发一种基于整数线性规划 (ILP) 的配置引导优化（称为 AutoTM），以最佳方式将实时张量分配和移动到 DRAM 或 NVDIMM。我们的方法可以用 PMM 替换系统 50% 到 80% 的 DRAM，而几何平均性能仅损失 27.7%。与首次接触 NUMA 相比，这是一个显着的改进，后者损失了 71.9% 的性能。与使用 DRAM 作为超大型网络的硬件控制缓存相比，所提出的基于 ILP 的同步调度技术还提供了 2 倍的性能。
Memory capacity is a key bottleneck for training large scale neural networks. Intel® Optane#8482; DC PMM (persistent memory modules) which are available as NVDIMMs are a disruptive technology that promises significantly higher read bandwidth than traditional SSDs at a lower cost per bit than traditional DRAM. In this work we show how to take advantage of this new memory technology to minimize the amount of DRAM required without compromising performance significantly. Specifically, we take advantage of the static nature of the underlying computational graphs in deep neural network applications to develop a profile guided optimization based on Integer Linear Programming (ILP) called AutoTM to optimally assign and move live tensors to either DRAM or NVDIMMs. Our approach can replace 50% to 80% of a system's DRAM with PMM while only losing a geometric mean 27.7% performance. This is a significant improvement over first-touch NUMA, which loses 71.9% of performance. The proposed ILP based synchronous scheduling technique also provides 2x performance over using DRAM as a hardware-controlled cache for very large networks.