A flexible hash table design for 10GBPS key-value stores on FPGAS
A flexible hash table design for 10GBPS key-value stores on FPGAS
复制标题
用于 FPGA 上 10GBPS 键值存储的灵活哈希表设计
DOI：
--
复制
发表时间：
2013
期刊：
影响因子：
--
通讯作者：
K. Vissers
中科院分区：
文献类型：
--
作者：
Z. István;G. Alonso;Michaela Blott;K. Vissers
通用Web基础设施依赖于分布式主存键值存储来减少数据库的访问负载，从而提高Web站点的性能和可伸缩性。由于标准云服务器为此类横向扩展工作负载提供了次线性可扩展性和降低的能效，因此我们研究了一种新型的基于FPGA的键值存储架构，该架构可以提供一致的10Gbps吞吐量。在本文中，我们提出了一种新的哈希表的设计，它形成了这一体系结构的核心部分。全流水线设计可以通过部署并发机制来处理哈希冲突，从而保持一致的10Gbps线速性能。我们解决的问题，如支持广泛的密钥大小，而不拖延管道通过仔细匹配的查找时间与数据包接收时间。最后，该设计基于一个可扩展的架构，可以很容易地参数化，以不同的访问速度和延迟与不同的内存类型。我们在memcached原型中部署了这个哈希表，以索引24千兆位外部DDR3 DRAM中的200万个条目，同时维持每秒1300万个UDP二进制编码的memcached数据包请求，这是memcached在10 Gbps链路上可以实现的最大数据包速率。
Common web infrastructure relies on distributed main memory key-value stores to reduce access load on databases, thereby improving both performance and scalability of web sites. As standard cloud servers provide sub-linear scalability and reduced power efficiency to these kinds of scale-out workloads, we have investigated a novel dataflow architecture for key-value stores with the aid of FPGAs which can deliver consistent 10Gbps throughput. In this paper, we present the design of a novel hash table which forms the centre piece of this dataflow architecture. The fully pipelined design can sustain consistent 10Gbps line-rate performance by deploying a concurrent mechanism to handle hash collisions. We address problems such as support for a broad range of key sizes without stalling the pipeline through careful matching of lookup time with packet reception time. Finally, the design is based on a scalable architecture that can be easily parametrized to work with different memory types operating at different access speeds and latencies. We deployed this hash table in a memcached prototype to index 2 million entries in 24GBytes of external DDR3 DRAM while sustaining 13 million requests per second for UDP binary encoded memcached packets which is the maximum packet rate that can be achieved with memcached on a 10Gbps link.