面向近数据计算的键值存储关键技术研究

基于LSM-tree的键值存储在非结构化数据存储中体现了较高性能，但是，compaction操作引起数据写放大，性能低下，这主要受限于以计算为中心的计算模式。近数据计算以数据为中心，将compaction任务迁移至数据存储位置，减少compaction代价，提升系统性能。为实现面向近数据计算的键值存储系统，针对近数据计算与键值存储的适配性；拟提出适配近数据计算的键值存储接口、基于协同感知的多级任务迁移策略；拟提出服务于键值存储的近数据计算体系结构原型及多级并行方法；针对近数据计算架构的有效性，拟提出基于负载特征和介质状态协同感知的方法，以及基于感知的主动式写回缓存方法；针对近数据计算架构下资源需求的差异性，拟提出基于任务与数据多样化的资源调度与管理方法。.项目预期成果将实现一种面向近数据计算架构下具有感知与自适应能力的键值存储系统，其将为大数据分析与处理提供强有力的支撑。

Log-structured merge-tree-based key-value stores achieve high performance for unstructured data storage. However, the compaction process causes write amplification, which affects the performance. This problem is mainly caused by the computing-centric model. Near-data processing is a data-centric model that partitions and offloads compaction tasks to where the key-value data resides. We aim to improve the performance of storage systems by exploring the near-data processing paradigm and optimizing compaction operations. This project proposes a key-value storage system with a near-data processing framework. To achieve the adaptability between a key-value store and the proposed framework, there are three research thrusts: (1) a key-value storage interface that is suitable for near-data computing; (2) a multi-level task offloading strategy that employs a collaborative-awareness method; and (3) a prototype of near-data computing architecture with a multi-level parallel method that is tailored for key-value stores. To achieve efficiency for near-data processing framework, this project proposes (1) collaborative awareness cache strategy based on real-time status in flash memory and I/Os behaviors in workload; and (2) an efficiently active write-back cache strategy based on the awareness semantics. To achieve differential requirements of heterogeneous resources, we propose the resources scheduling strategy and management based on the diversification of tasks and data..The project aims to develop a high-performance key-value storage system which is aware of and adaptive to system runtime status and achieves near-data computing. The proposed key-value storage and our novel methods will significantly improve the system performance and resource utilization for big data analytics and processing.