CSR: Medium: Collaborative Research: Data Center Scale Programmable Storage

从历史上看，应用程序通常通过协议、抽象和接口与持久性存储系统进行交互。协议定义了一系列可以在存储上执行的操作，例如读或写，并具有隐式的抽象模型，例如块或对象。通用或专用存储网络等接口传输协议消息。虽然协议和接口简化了存储系统的设计，但由于使用抽象模型和限制操作而影响性能。新兴的存储类内存具有低延迟和粒度操作，大大增加了协议、抽象和接口开销的相对影响。该项目将存储接口重新想象为可编程存储，其中定义的协议涉及将封装的程序发送到实际存储，并在其中应用它们的效果。这将提高分布式系统和存储应用程序的性能，这两者都是Internet应用程序的核心。这个项目中的智力挑战是新存储抽象的设计，确保它们适用于各种新的存储介质，并在提高性能的同时维护安全性。我们正在扩展现有的存储系统，以包括可编程存储的特性，同时还支持分布式系统使用的低级组件的新共识，例如内存级别的持久日志和事务操作。该项目使用现有的运行时代码生成框架来确保可编程接口生成的可移植代码也是高效的。研究工作的一部分是在行业和学术研究人员之间就我们提出的存储抽象的必要性和充分性达成共识，并开发教育材料来演示如何使用它们。

Historically, applications usually interface with persistent storage systems through protocols, abstractions and interfaces. Protocols define a series of actions that can be taken on the storage such as reading, or writing and have an implicit abstraction model such as blocks or objects. Interfaces such as general or dedicated storage networks transport protocol messages. While protocols and interfaces simplify storage system design, both impact performance by the use of abstraction models and limited operations. Emerging storage class memory has low latency and granular operations, greatly increasing the relative impact of the overhead of protocols, abstractions and interfaces. This project is re-imagining the storage interface as programmable storage, where the defined protocol involves sending encapsulated programs to the actual storage where their effect is applied. This will improve the performance of distributed systems as well as storage applications, both of which are central to Internet applications.The intellectual challenge in this project is the design of the new storage abstractions, insuring they are suitable for a variety of new storage media and that security is maintained while performance is improved. We are extending an existing storage system to include features of programmable storage while also supporting emerging consensus on low-level components used by distributed systems, such as persistent logs and transactional operations at the memory level. The project is using existing run-time code generation frameworks to insure that the programmable interface generates portable code that is also efficient. Part of the research effort is developing consensus among industry and academic researchers on the necessity and sufficiency of the storage abstractions we propose and developing educational materials to demonstrate how they should be used.