DLIO: A Data-Centric Benchmark for Scientific Deep Learning Applications
DLIO: A Data-Centric Benchmark for Scientific Deep Learning Applications
复制标题
DOI：
10.1109/ccgrid51090.2021.00018
复制
发表时间：
2021-05
期刊：
影响因子：
--
通讯作者：
H. Devarajan;Huihuo Zheng;Anthony Kougkas;Xian-He Sun;V. Vishwanath
中科院分区：
文献类型：
--
作者：
H. Devarajan;Huihuo Zheng;Anthony Kougkas;Xian-He Sun;V. Vishwanath
深度学习已被证明是各种任务的成功方法，它的流行导致了许多开源深度学习软件工具。深度学习已被应用于广泛的科学领域，如宇宙学，粒子物理学，计算机视觉，融合和天体物理学。科学家们已经做了大量的工作来优化深度学习框架的计算性能。然而，I/O性能却不能这么说。由于深度学习算法依赖于大数据量和多样性来有效准确地训练神经网络，因此I/O是大规模分布式深度学习训练的重要瓶颈。本研究旨在详细调查在阿贡领导力计算设施的Theta超级计算机上运行的各种科学深度学习工作负载的I/O行为。在本文中，我们提出了DLIO，一个新的代表性的基准套件的基础上建立的I/O分析选定的工作负载。DLIO可用于精确模拟现代科学深度学习应用程序的I/O行为。使用DLIO，应用程序开发人员和系统软件解决方案架构师可以识别其应用程序中潜在的I/O瓶颈，并指导优化以提高I/O性能，从而将训练时间缩短多达6.7倍。
Deep learning has been shown as a successful method for various tasks, and its popularity results in numerous open-source deep learning software tools. Deep learning has been applied to a broad spectrum of scientific domains such as cosmology, particle physics, computer vision, fusion, and astrophysics. Scientists have performed a great deal of work to optimize the computational performance of deep learning frameworks. However, the same cannot be said for I/O performance. As deep learning algorithms rely on big-data volume and variety to effectively train neural networks accurately, I/O is a significant bottleneck on large-scale distributed deep learning training. This study aims to provide a detailed investigation of the I/O behavior of various scientific deep learning workloads running on the Theta supercomputer at Argonne Leadership Computing Facility. In this paper, we present DLIO, a novel representative benchmark suite built based on the I/O profiling of the selected workloads. DLIO can be utilized to accurately emulate the I/O behavior of modern scientific deep learning applications. Using DLIO, application developers and system software solution architects can identify potential I/O bottlenecks in their applications and guide optimizations to boost the I/O performance leading to lower training times by up to 6.7x.