dCompaction: Delayed Compaction for the LSM-Tree
dCompaction: Delayed Compaction for the LSM-Tree
复制标题
DOI：
10.1007/s10766-016-0472-z
复制
发表时间：
2017-12
影响因子：
1.5
通讯作者：
Fengfeng Pan;Yinliang Yue;Jin Xiong
中科院分区：
文献类型：
--
作者：
Fengfeng Pan;Yinliang Yue;Jin Xiong
键值（KV）存储已成为当今数据中心中大规模应用的骨干。写入优化的数据结构，如日志结构合并树（LSM树）及其变体，广泛用于KV存储系统，如BigTable和RocksDB。传统的LSM树将KV项组织成多个连续的较大组件，并使用压缩将KV项从一个较小组件推到另一个相邻的较大组件，直到KV项到达最大组件。不幸的是，当前的压缩方案由于重复的KV项读取和写入而引起显著的写入放大，并且然后导致差的吞吐量。我们提出了一个新的压缩方案，延迟压缩（dCompaction），减少写放大。dCompaction推迟一些压缩并将它们收集到下面的压缩中。通过这种方式，它避免了压缩期间的KV项读取和写入，从而提高了基于LSM树的KV存储的吞吐量。我们在RocksDB上实现了DCompactionTM，并进行了大量的实验。使用YCSB框架验证表明，与RocksDBdCompaction相比，写性能提高了约30%，读性能也相当。
Key-value (KV) stores have become a backbone of large-scale applications in today’s data centers. Write-optimized data structures like the Log-Structured Merge-tree (LSM-tree) and their variants are widely used in KV storage systems like BigTable and RocksDB. Conventional LSM-tree organizes KV items into multiple, successively larger components, and uses compaction to push KV items from one smaller component to another adjacent larger component until the KV items reach the largest component. Unfortunately, current compaction scheme incurs significant write amplification due to repeated KV item reads and writes, and then results in poor throughput. We propose a new compaction scheme,delayed compaction(dCompaction), that decreases write amplification. dCompaction postpones some compactions and gather them into the following compaction. In this way, it avoids KV item reads and writes during compaction, and consequently improves the throughput of LSM-tree based KV stores. We implementdCompactionon RocksDB, and conduct extensive experiments. Validation using YCSB framework shows that compared with RocksDBdCompactionhas about 30% write performance improvements and also comparable read performance.