Matrix Engines for High Performance Computing: A Paragon of Performance or Grasping at Straws?
Matrix Engines for High Performance Computing: A Paragon of Performance or Grasping at Straws?
复制标题
DOI：
10.1109/ipdps49936.2021.00114
复制
发表时间：
2020-10
期刊：
影响因子：
--
通讯作者：
Jens Domke;Emil Vatai;Aleksandr Drozd;Peng Chen;Yosuke Oyama;Lingqi Zhang;Shweta Salaria;Daichi Mukunoki;Artur Podobas;M. Wahib;S. Matsuoka
中科院分区：
文献类型：
--
作者：
Jens Domke;Emil Vatai;Aleksandr Drozd;Peng Chen;Yosuke Oyama;Lingqi Zhang;Shweta Salaria;Daichi Mukunoki;Artur Podobas;M. Wahib;S. Matsuoka
矩阵引擎或不同形式和亲和力的单位正在成为现代处理器中的现实；即高性能的Linpack，也希望HPC社区唤醒热情，因此我们的目标是通过访问Matrix引擎来确定HPC和机器学习应用程序的实际额外好处。对软件堆栈，代理应用程序和基准和历史批处理记录的深入调查。缓解热情，我们还概述了这些密集的矩阵 - 刺激引擎的机会，如果它们可以免费。
Matrix engines or units, in different forms and affinities, are becoming a reality in modern processors; CPUs and otherwise. The current and dominant algorithmic approach to Deep Learning merits the commercial investments in these units, and deduced from the No. 1 benchmark in supercomputing, namely High Performance Linpack, one would expect an awakened enthusiasm by the HPC community, too. Hence, our goal is to identify the practical added benefits for HPC and machine learning applications by having access to matrix engines. For this purpose, we perform an in-depth survey of software stacks, proxy applications and benchmarks, and historical batch job records. We provide a cost-benefit analysis of matrix engines, both asymptotically and in conjunction with state-of-the-art processors. While our empirical data will temper the enthusiasm, we also outline opportunities to “misuse” these dense matrix-multiplication engines if they come for free.