SHF:Small: Distributed Key Manager for higher functionality KV Stores

键值（KV）存储已广泛应用于各种企业应用，如在线零售、大数据分析、社交网络等。键值固态设备直接从设备提供键值接口，旨在降低软件开销并减少此类应用程序的I/O放大。随着KV器件的出现，需要进行新的研究，在这些新型器件的基础上构建更高层的系统。该项目计划解决研究问题，使这些新型KV装置得到更广泛的采用。(1)当前的KV存储依赖于复制来提供容错。虽然复制简单而有效，但它会带来高存储开销和高写放大。开发替代的低成本容错方案将支持更广泛地采用KV设备。(2)许多应用程序需要支持范围查询。在KV设备上提供有效的键索引可以增强这些应用。(3)当前可用的系统与块设备接口，并且有关KV记录的语义信息仅在应用程序中保留。该项目建议利用千伏设备的语义信息，为千伏存储提供额外的功能和优化，如近存储处理。这个项目有以下目标。(1)设计、开发和评估分布式密钥管理器，以在KV设备之上重新构建KV存储。(2)研究在分布式密钥管理器中提取信息和组织元数据的机制。(3)研究有效处理KV设备附近存储数据的机制。(4)探索在分布式密钥管理器中构建增强功能和接口的选项。(5)调查在分布式密钥管理器中支持这个新功能的成本效益权衡。为了实现项目目标，研究人员计划采用间接、新颖的索引、近存储的智能处理和跨设备的协作。该项目有望提供新的增强功能，可以在KV设备之上分层，并指出构建应用程序的新方法，以减少软件开销。所提出的研究预计将导致开发利用具有新接口的新兴存储设备的新方法，使近存储处理应用取得进展。这些进步将带来重要的社会效益，如大数据处理，促进新的科学成就和理解。教育方面的影响将包括培养研究生和本科生有价值的研究技能，同时推动计算机体系结构和分布式存储系统的发展，为技术劳动力做出贡献。少数民族学生将参与该项目。与该项目有关的计划中的工业合作预计将加速已开发系统的商业采用。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Key-value (KV) stores have been widely deployed in a variety of enterprise applications such as online retail, big-data analytics, social networks and others. Key-Value Solid-State Devices provide a key-value interface directly from devices aiming at lowering software overheads and reducing I/O amplification for such applications. With the advent of KV devices, new research is needed to architect higher-layer systems on top of these novel devices. This project plans to address the research issues to enable wider adoption of these novel KV devices. (1) Current KV stores rely on replication for providing fault tolerance. While replication is simple and effective, it incurs high storage overhead and high write amplification. Developing alternate lower-cost fault-tolerance options would support wider adoption of KV devices. (2) Many applications require support for range queries. Providing efficient indexing of keys at the KV devices could enhance these applications. (3) Currently available systems interface with block devices, and the semantic information about the KV records is only retained within the application. This project proposes to exploit the semantic information at the KV devices to provide additional functionality and optimizations for a KV store, such as near-storage processing. This project has the following objectives. (1) Design, develop and evaluate a distributed key manager to rearchitect KV stores on top of KV devices. (2) Investigate mechanisms for extracting information and organizing metadata within the distributed key manager. (3) Investigate mechanisms for efficient processing of stored data near the KV devices. (4) Explore options for architecting enhanced functionality and interfaces within the distributed key manager. (5) Investigate cost-benefit tradeoffs in supporting this new functionality within the distributed key manager. In order to achieve the project goals, the researchers plan to employ indirection, novel indexing, intelligent processing near storage and collaboration across devices. The project is expected to provide new enhancements that can be layered on top of KV devices and point to new ways of architecting applications to reduce software overheads. The proposed research is expected to lead to the development of new ways of utilizing emerging storage devices with new interfaces, enable advances in near-storage processing applications. These advances will enable important societal benefits such as big-data processing, facilitating new scientific achievements and understanding. Educational impact will include training graduate and undergraduate students with valuable research skills while advancing the state of the art in computer architecture and distributed storage systems, contributing to the technology workforce. Minority students will be involved in the project. The planned industrial collaborations related to the project are expected to hasten commercial adoption of developed systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.