Towards Unified Data and Lifecycle Management for Deep Learning
Towards Unified Data and Lifecycle Management for Deep Learning
复制标题
DOI：
10.1109/icde.2017.112
复制
发表时间：
2016-11
期刊：
影响因子：
--
通讯作者：
Hui Miao;Ang Li;L. Davis;A. Deshpande
中科院分区：
文献类型：
--
作者：
Hui Miao;Ang Li;L. Davis;A. Deshpande
深度学习在许多重要领域改善了最先进的结果，近年来一直是许多研究的主题，导致了几个促进深度学习的系统的发展。然而，目前的系统主要集中在模型建立和培训阶段，而数据管理、模型共享和生命周期管理等问题在很大程度上被忽视。深度学习建模生命周期生成一组丰富的数据构件，例如，学习的参数和训练日志，它包括几个频繁执行的任务，例如，理解模型行为和尝试新模型。处理这样的构件和任务很麻烦，并且很大程度上留给了用户。本文描述了我们对深度学习的数据和生命周期管理系统的愿景和实施。首先，我们从深度学习建模者经常执行的任务中概括出模型探索和模型枚举查询，并在SQL的启发下提出了一种高级领域特定语言(DSL)，以提高抽象级别，从而加快建模过程。为了管理各种各样的数据伪像，特别是大量的检查点浮点参数，我们设计了一个新的模型版本控制系统(DLV)和一个读优化的参数归档存储系统(PAS)，该系统最大限度地减少了存储空间，并在最小精度损失的情况下加快了查询工作量。PAS通过分别存储较低有效位来以多分辨率方式使用增量对版本化模型进行存档，并采用了一种新的渐进查询(推理)评估算法。第三，提出了在共检索约束下利用增量对版本化模型进行归档的有效算法。我们在几个计算机视觉领域的真实数据集上进行了大量的实验，以表明所提出的技术的有效性。
Deep learning has improved state-of-the-art results in many important fields, and has been the subject of much research in recent years, leading to the development of several systems for facilitating deep learning. Current systems, however, mainly focus on model building and training phases, while the issues of data management, model sharing, and lifecycle management are largely ignored. Deep learning modeling lifecycle generates a rich set of data artifacts, e.g., learned parameters and training logs, and it comprises of several frequently conducted tasks, e.g., to understand the model behaviors and to try out new models. Dealing with such artifacts and tasks is cumbersome and largely left to the users. This paper describes our vision and implementation of a data and lifecycle management system for deep learning. First, we generalize model exploration and model enumeration queries from commonly conducted tasks by deep learning modelers, and propose a high-level domain specific language (DSL), inspired by SQL, to raise the abstraction level and thereby accelerate the modeling process. To manage the variety of data artifacts, especially the large amount of checkpointed float parameters, we design a novel model versioning system (dlv), and a read-optimized parameter archival storage system (PAS) that minimizes storage footprint and accelerates query workloads with minimal loss of accuracy. PAS archives versioned models using deltas in a multi-resolution fashion by separately storing the less significant bits, and features a novel progressive query (inference) evaluation algorithm. Third, we develop e cient algorithms for archiving versioned models using deltas under co-retrieval constraints. We conduct extensive experiments over several real datasets from computer vision domain to show the e ciency of the proposed techniques.