RStore: A Distributed Multi-Version Document Store
RStore: A Distributed Multi-Version Document Store
复制标题
DOI：
10.1109/icde.2018.00043
复制
发表时间：
2018-02
期刊：
影响因子：
--
通讯作者：
Souvik Bhattacherjee;A. Deshpande
中科院分区：
文献类型：
--
作者：
Souvik Bhattacherjee;A. Deshpande
我们解决了在分布式环境中紧凑地存储大量关键字文档或记录集合的版本(快照)的问题，同时有效地回答了关于这些文档或记录的各种检索查询，包括检索完整或部分版本以及特定关键字的演变历史。我们推动了各种应用领域对这种系统日益增长的需求，仔细探索了构建这种系统的设计空间和各种存储-计算-检索的权衡，并讨论了不同的存储布局如何影响这些权衡。我们提出了一种新颖的系统体系结构，它满足了这种系统的关键需求，并提供了简单的调优旋钮，允许适应特定的数据和查询工作负载。我们的系统旨在充当分布式键值存储的顶层，该存储包含原始数据和任何索引。我们设计了新颖的离线存储布局算法来有效地划分数据，以最大限度地减少存储成本，同时保持较低的检索成本。我们还提出了一个在线算法来处理添加到系统中的新版本。通过在大型数据集上的大量实验，我们证明了我们的系统以大多数实际场景所需的规模运行，并且通常比标准基准(包括基于增量的存储引擎)高出一个数量级。
We address the problem of compactly storing a large number of versions (snapshots) of a collection of keyed documents or records in a distributed environment, while efficiently answering a variety of retrieval queries over those, including retrieving full or partial versions, and evolution histories for specific keys. We motivate the increasing need for such a system in a variety of application domains, carefully explore the design space for building such a system and the various storage-computation-retrieval trade-offs, and discuss how different storage layouts influence those trade-offs. We propose a novel system architecture that satisfies the key desiderata for such a system, and offers simple tuning knobs that allow adapting to a specific data and query workload. Our system is intended to act as a layer on top of a distributed key-value store that houses the raw data as well as any indexes. We design novel off-line storage layout algorithms for efficiently partitioning the data to minimize the storage costs while keeping the retrieval costs low. We also present an online algorithm to handle new versions being added to system. Using extensive experiments on large datasets, we demonstrate that our system operates at the scale required in most practical scenarios and often outperforms standard baselines, including a delta-based storage engine, by orders-of-magnitude.