SHF:Small: Distributed Key Manager for higher functionality KV Stores

键值（KV）存储已被广泛部署在各种企业应用中，诸如在线零售、大数据分析、社交网络等。键值固态设备直接从设备提供键值接口，旨在降低软件开销并减少此类应用的I/O放大。随着KV设备的出现，需要新的研究来在这些新设备之上构建更高层的系统。该项目计划解决研究问题，以使这些新型KV设备得到更广泛的采用。(1)当前KV存储依赖于复制来提供容错。虽然复制简单有效，但它会导致高存储开销和高写入放大。开发替代的低成本容错选项将支持更广泛地采用KV设备。(2)许多应用程序需要支持范围查询。在KV设备处提供密钥的高效索引可以增强这些应用。(3)当前可用的系统与块设备接口，并且关于KV记录的语义信息仅保留在应用程序内。该项目建议利用KV设备的语义信息为KV存储提供额外的功能和优化，例如近存储处理。 该项目有以下目标。(1)设计、开发和评估分布式密钥管理器，在KV设备上重新构建KV存储。(2)研究在分布式密钥管理器中提取信息和组织元数据的机制。(3)调查KV设备附近存储数据的有效处理机制。(4)探索在分布式密钥管理器中构建增强功能和接口的选项。(5)研究在分布式密钥管理器中支持此新功能的成本效益权衡。为了实现项目目标，研究人员计划采用间接、新颖的索引、存储附近的智能处理以及跨设备的协作。该项目预计将提供新的增强功能，可以在KV设备之上分层，并指出构建应用程序以减少软件开销的新方法。预计拟议的研究将导致开发利用具有新接口的新兴存储设备的新方法，使近存储处理应用程序取得进展。这些进步将带来重要的社会效益，如大数据处理，促进新的科学成就和理解。教育影响将包括培养研究生和本科生的宝贵研究技能，同时推进计算机体系结构和分布式存储系统的最新技术，为技术劳动力做出贡献。少数民族学生将参与该项目。与该项目相关的计划中的工业合作预计将加速已开发系统的商业应用。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Key-value (KV) stores have been widely deployed in a variety of enterprise applications such as online retail, big-data analytics, social networks and others. Key-Value Solid-State Devices provide a key-value interface directly from devices aiming at lowering software overheads and reducing I/O amplification for such applications. With the advent of KV devices, new research is needed to architect higher-layer systems on top of these novel devices. This project plans to address the research issues to enable wider adoption of these novel KV devices. (1) Current KV stores rely on replication for providing fault tolerance. While replication is simple and effective, it incurs high storage overhead and high write amplification. Developing alternate lower-cost fault-tolerance options would support wider adoption of KV devices. (2) Many applications require support for range queries. Providing efficient indexing of keys at the KV devices could enhance these applications. (3) Currently available systems interface with block devices, and the semantic information about the KV records is only retained within the application. This project proposes to exploit the semantic information at the KV devices to provide additional functionality and optimizations for a KV store, such as near-storage processing. This project has the following objectives. (1) Design, develop and evaluate a distributed key manager to rearchitect KV stores on top of KV devices. (2) Investigate mechanisms for extracting information and organizing metadata within the distributed key manager. (3) Investigate mechanisms for efficient processing of stored data near the KV devices. (4) Explore options for architecting enhanced functionality and interfaces within the distributed key manager. (5) Investigate cost-benefit tradeoffs in supporting this new functionality within the distributed key manager. In order to achieve the project goals, the researchers plan to employ indirection, novel indexing, intelligent processing near storage and collaboration across devices. The project is expected to provide new enhancements that can be layered on top of KV devices and point to new ways of architecting applications to reduce software overheads. The proposed research is expected to lead to the development of new ways of utilizing emerging storage devices with new interfaces, enable advances in near-storage processing applications. These advances will enable important societal benefits such as big-data processing, facilitating new scientific achievements and understanding. Educational impact will include training graduate and undergraduate students with valuable research skills while advancing the state of the art in computer architecture and distributed storage systems, contributing to the technology workforce. Minority students will be involved in the project. The planned industrial collaborations related to the project are expected to hasten commercial adoption of developed systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.