Dremel
Dremel
复制标题
DOI：
10.1145/3547353.3530970
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
S. Melnik;Andrey Gubarev;Jing Jing Long-Jing;Geoffrey Romer;Shiva Shivakumar;Matt Tolton;Theo Vassilakis;Hossein Ahmadi;Daniel P. Delorey;Slava Min;Mosha Pasumansky-;J. Shute
中科院分区：
文献类型：
--
作者：
S. Melnik;Andrey Gubarev;Jing Jing Long-Jing;Geoffrey Romer;Shiva Shivakumar;Matt Tolton;Theo Vassilakis;Hossein Ahmadi;Daniel P. Delorey;Slava Min;Mosha Pasumansky-;J. Shute
像RocksDB这样基于LSM树的键值存储被广泛用于支持许多应用程序。然而，配置RocksDB实例是具有挑战性的，原因如下：1）RocksDB有大量的参数空间要配置; 2）参数之间存在固有的权衡和依赖关系; 3）最佳配置取决于工作负载和硬件;以及4）评估配置是耗时的。以前的工作与处理维数灾难，捕捉参数之间的关系，适应配置的工作负载和硬件，并快速评估的斗争。我们提出了一个系统，Dremel，自适应和快速配置RocksDB的策略的基础上的多臂强盗模型。为了处理大的参数空间，我们建议使用融合的功能，编码特定领域的知识，作为一个紧凑和强大的表示配置。为了适应工作负载和硬件条件，我们建立了一个在线的强盗模型，以确定最佳的配置。为了快速评估，我们启用了多保真度评估和置信上限采样，以加快配置搜索。Dremel不仅实现了比默认配置高出2.61倍的IOPS和减少57%的延迟，而且在相同或更少的时间预算下，在18种不同设置上实现了比以前工作高出63%的改进。本文是节略本。
LSM-tree-based key-value stores like RocksDB are widely used to support many applications. However, configuring a RocksDB instance is challenging for the following reasons: 1) RocksDB has a massive parameter space to configure; 2) there are inherent trade-offs and dependencies between parameters; 3) optimal configurations are dependent on workload and hardware; and 4) evaluating configurations is time-consuming. Prior works struggle with handling the curse of dimensionality, capturing relationships between parameters, adapting configurations to workload and hardware, and evaluating quickly. We present a system, Dremel, to adaptively and quickly configure RocksDB with strategies based on the Multi-Armed Bandit model. To handle the large parameter space, we propose using fused features, which encode domain-specific knowledge, to work as a compact and powerful representation for configurations. To adapt to the workload and hardware, we build an online bandit model to identify the best configuration. To evaluate quickly, we enable multi-fidelity evaluation and upper-confidence-bound sampling to speed up configuration search. Dremel not only achieves up to ×2.61 higher IOPS and 57% less latency than default configurations but also achieves up to 63% improvement over prior works on 18 different settings with the same or smaller time budget. This paper is an abridged version.