Empirical Analysis and Modeling of Compute Times of CNN Operations on AWS Cloud
Empirical Analysis and Modeling of Compute Times of CNN Operations on AWS Cloud
复制标题
DOI：
10.1109/iiswc50251.2020.00026
复制
发表时间：
2020-10
期刊：
影响因子：
--
通讯作者：
Ubaid Ullah Hafeez;Anshul Gandhi
中科院分区：
文献类型：
--
作者：
Ubaid Ullah Hafeez;Anshul Gandhi
鉴于卷积神经网络（CNN）在图像分类应用中的广泛使用，云提供商现在通常会提供几个配备GPU的实例，这些实例具有不同的价格点和硬件规格。从从业者的角度来看，给定任意CNN，应该使用哪个GPU实例来最小化模型训练时间和/或租赁成本并不明显。本文介绍了Ceer，这是一种模型驱动的方法，用于确定任何给定CNN的最佳GPU实例。基于对各种CNN的操作级经验分析，我们为繁重的GPU操作（其中输入大小是一个关键特征）开发了回归模型，并为轻型GPU和CPU操作使用了样本中值估计器。为了估计CPU和GPU之间的通信开销，特别是在多GPU训练的情况下，我们开发了一个模型，将此通信开销与CNN中的模型参数数量相关联。AWS Cloud上的评估结果表明，Ceer可以准确预测CNN的训练时间和成本（平均预测误差小于5%），与采用最便宜或最新一代GPU实例的简单策略相比，可以节省36%-44%的成本。
Given the widespread use of Convolutional Neural Networks (CNNs) in image classification applications, cloud providers now routinely offer several GPU-equipped instances with varying price points and hardware specifications. From a practitioner's perspective, given an arbitrary CNN, it is not obvious which GPU instance should be employed to minimize the model training time and/or rental cost. This paper presents Ceer, a model-driven approach to determine the optimal GPU instance(s) for any given CNN. Based on an operation-level empirical analysis of various CNNs, we develop regression models for heavy GPU operations (where input size is a key feature) and employ the sample median estimator for light GPU and CPU operations. To estimate the communication overhead between CPU and GPU(s), especially in the case of multi-GPU training, we develop a model that relates this communication overhead to the number of model parameters in the CNN. Evaluation results on AWS Cloud show that Ceer can accurately predict training time and cost (less than 5% average prediction error) across CNNs, enabling 36% −44% cost savings over simpler strategies that employ the cheapest or the latest generation GPU instances.