Serving Deep Learning Models with Deduplication from Relational Databases
Serving Deep Learning Models with Deduplication from Relational Databases
复制标题
DOI：
10.14778/3547305.3547325
复制
发表时间：
2022-01
期刊：
影响因子：
--
通讯作者：
Lixi Zhou;Jiaqing Chen;Amitabh Das;Hong Min;Lei Yu;Ming Zhao;Jia Zou
中科院分区：
文献类型：
--
作者：
Lixi Zhou;Jiaqing Chen;Amitabh Das;Hong Min;Lei Yu;Ming Zhao;Jia Zou
从关系数据库中提供深度学习模型带来了巨大的好处。首先，从数据库中提取的特征不需要转移到任何解耦的深度学习系统进行推理，因此可以显著降低系统管理开销。其次，在关系数据库中，数据管理沿着存储层次结构与查询处理完全集成，因此即使工作集大小超过可用内存，它也可以继续模型服务。应用模型重复数据删除可以大大减少存储空间、内存占用、缓存未命中和推理延迟。然而，现有的数据去重技术不适用于服务于关系数据库中的应用的深度学习模型。它们没有考虑对模型推理精度的影响以及张量块和数据库页面之间的不一致性。这项工作提出了协同存储优化技术的重复检测，页面打包，缓存，以提高数据库系统的模型服务。评估结果表明，我们提出的技术显着提高了存储效率和模型推理延迟，并在目标场景中优于现有的深度学习框架。
Serving deep learning models from relational databases brings significant benefits. First, features extracted from databases do not need to be transferred to any decoupled deep learning systems for inferences, and thus the system management overhead can be significantly reduced. Second, in a relational database, data management along the storage hierarchy is fully integrated with query processing, and thus it can continue model serving even if the working set size exceeds the available memory. Applying model deduplication can greatly reduce the storage space, memory footprint, cache misses, and inference latency. However, existing data deduplication techniques are not applicable to the deep learning model serving applications in relational databases. They do not consider the impacts on model inference accuracy as well as the inconsistency between tensor blocks and database pages. This work proposed synergistic storage optimization techniques for duplication detection, page packing, and caching, to enhance database systems for model serving. Evaluation results show that our proposed techniques significantly improved the storage efficiency and the model inference latency, and outperformed existing deep learning frameworks in targeting scenarios.