MagmaDNN: Towards High-Performance Data Analytics and Machine Learning for Data-Driven Scientific Computing
MagmaDNN: Towards High-Performance Data Analytics and Machine Learning for Data-Driven Scientific Computing
复制标题
DOI：
10.1007/978-3-030-34356-9_37
复制
发表时间：
2019-06
期刊：
影响因子：
--
通讯作者：
Daniel Nichols;N. Tomov;Frank Betancourt;S. Tomov;Kwai Wong;J. Dongarra
中科院分区：
文献类型：
--
作者：
Daniel Nichols;N. Tomov;Frank Betancourt;S. Tomov;Kwai Wong;J. Dongarra
在本文中，我们介绍了一种新的数据分析和机器学习（ML）框架的开发工作，称为MagmaDNN。我们的主要目标是为运行在当前和即将到来的异构多核gpu加速架构上的科学应用程序提供可扩展的高性能数据分析和机器学习解决方案。为此，由于所需的许多功能都是基于标准线性代数（LA）例程，因此我们设计了MagmaDNN，以从MAGMA库中获得其性能。这种紧密的集成提供了基本的（可扩展的高性能的）LA例程，作为MagmaDNN的后端。我们提出了一些关于使用深度神经网络（DNN）的机器学习的性能和可扩展性的设计问题，以及用于克服这些问题的MagmaDNN设计。特别是，MagmaDNN使用了来自密集LA领域的成熟的HPC技术，包括基于任务的并行化、DAG表示、调度、混合精度算法、异步求解器和自调超参数优化。我们将介绍这些技术，以及它们的结合和使用，以超越目前可用的其他框架。
In this paper, we present work towards the development of a new data analytics and machine learning (ML) framework, called MagmaDNN. Our main goal is to provide scalable, high-performance data analytics and ML solutions for scientific applications running on current and upcoming heterogeneous many-core GPU-accelerated architectures. To this end, since many of the functionalities needed are based on standard linear algebra (LA) routines, we designed MagmaDNN to derive its performance power from the MAGMA library. The close integration provides the fundamental (scalable high-performance) LA routines available in MAGMA as a backend to MagmaDNN. We present some design issues for performance and scalability that are specific to ML using Deep Neural Networks (DNN), as well as the MagmaDNN designs towards overcoming them. In particular, MagmaDNN uses well established HPC techniques from the area of dense LA, including task-based parallelization, DAG representations, scheduling, mixed-precision algorithms, asynchronous solvers, and autotuned hyperparameter optimization. We illustrate these techniques and their incorporation and use to outperform other frameworks, currently available.