Endure: A Robust Tuning Paradigm for LSM Trees Under Workload Uncertainty
Endure: A Robust Tuning Paradigm for LSM Trees Under Workload Uncertainty
复制标题
DOI：
10.14778/3529337.3529345
复制
发表时间：
2021-10
期刊：
影响因子：
--
通讯作者：
Andrew Huynh;Harshal A. Chaudhari;Evimaria Terzi;Manos Athanassoulis
中科院分区：
文献类型：
--
作者：
Andrew Huynh;Harshal A. Chaudhari;Evimaria Terzi;Manos Athanassoulis
日志结构化合并树（LSM树）越来越多地用作多个数据系统背后的存储引擎，经常部署在云中。与其他数据库架构类似，LSM树考虑有关预期工作负载的信息（例如，读取与写入、点查询与范围查询），以通过调优来优化其性能。然而，由于现代应用程序的快速发展特性，在云等共享基础设施中运行会带来工作负载的不确定性。具有静态调优的系统降低了这种混合工作负载的可变性，因此提供了不一致和总体次优的性能。为了解决这个问题，我们引入了Endure -一个新的范例，用于在工作负载不确定的情况下调整LSM树。具体来说，我们专注于压缩策略，大小比例和内存分配的选择对整体性能的影响。Endure考虑了吞吐量最大化问题的一个鲁棒公式，并建议在每个预期工作负载的邻域内最大化最坏情况下的吞吐量。此外，不确定性调谐参数控制该邻域的大小，从而允许输出调谐是保守的或乐观的。通过基于模型和广泛的实验评估的耐力在国家的最先进的基于LSM的存储引擎，RocksDB，我们表明，强大的调整方法始终优于经典的调整策略。Endure输出的稳健调整可在存在不确定性的情况下将吞吐量提高5倍。另一方面，当观察到的工作负载与预期工作负载完全匹配时，Endure调优的性能损失可以忽略不计。
Log-Structured Merge trees (LSM trees) are increasingly used as the storage engines behind several data systems, frequently deployed in the cloud. Similar to other database architectures, LSM trees consider information about the expected workload (e.g., reads vs. writes, point vs. range queries) to optimize their performance via tuning. However, operating in a shared infrastructure like the cloud comes with workload uncertainty due to the fast-evolving nature of modern applications. Systems with static tuning discount the variability of such hybrid workloads and hence provide an inconsistent and overall suboptimal performance. To address this problem, we introduce Endure - a new paradigm for tuning LSM trees in the presence of workload uncertainty. Specifically, we focus on the impact of the choice of compaction policies, size ratio, and memory allocation on the overall performance. Endure considers a robust formulation of the throughput maximization problem and recommends a tuning that maximizes the worst-case throughput over the neighborhood of each expected workload. Additionally, an uncertainty tuning parameter controls the size of this neighborhood, thereby allowing the output tunings to be conservative or optimistic. Through both model-based and extensive experimental evaluations of Endure in the state-of-the-art LSM-based storage engine, RocksDB, we show that the robust tuning methodology consistently outperforms classical tuning strategies. The robust tunings output by Endure lead up to a 5X improvement in throughput in the presence of uncertainty. On the flip side, Endure tunings have negligible performance loss when the observed workload exactly matches the expected one.