High-Speed Query Processing over High-Speed Networks
High-Speed Query Processing over High-Speed Networks
复制标题
DOI：
10.14778/2856318.2856319
复制
发表时间：
2015-02
期刊：
影响因子：
--
通讯作者：
Wolf Rödiger;Tobias Mühlbauer;A. Kemper;Thomas Neumann
中科院分区：
文献类型：
--
作者：
Wolf Rödiger;Tobias Mühlbauer;A. Kemper;Thomas Neumann
现代数据库群集需要两个级别的网络：在大型和多个服务器中连接CPU和NUMA区域。这两种类型的网络之间的巨大性能差距用于减慢分布式查询处理，以至于一组机器的性能实际上比单个多核服务器更糟。集群的主要记忆容量仍然是这种规模淘汰的唯一优势。高速互连（例如，Infiniband）的经济可行性大大缩小了这一性能差距。但是，仅在分布式查询引擎不变时，Infiniband的较高网络带宽并不能像预期的那样提高查询性能。分布式查询处理的可扩展性受到TCP开销的损害，由于不协调的通信而引起的开关争议以及经典交换运算符模型的僵化性导致的负载不平衡。本文介绍了分布式查询引擎的蓝图，该引擎通过整体考虑两个级别的网络来解决这些问题。它由两个部分组成：首先，混合并行性，区分局部和分布的并行性，以更好地在核心和服务器的数量中可扩展性。其次，使用远程直接内存访问（RDMA）和低延迟网络调度进行了针对分析数据库工作负载量身定制的新型通信多路复用器，用于高速通信，几乎没有CPU开销。使用TPC-H基准测试的超级数据库系统中的广泛评估表明，我们的整体方法确实可以通过高速网络进行高速查询处理。
Modern database clusters entail two levels of networks: connecting CPUs and NUMA regions inside a single server in the small and multiple servers in the large. The huge performance gap between these two types of networks used to slow down distributed query processing to such an extent that a cluster of machines actually performed worse than a single many-core server. The increased main-memory capacity of the cluster remained the sole benefit of such a scale-out. The economic viability of high-speed interconnects such as InfiniBand has narrowed this performance gap considerably. However, InfiniBand's higher network bandwidth alone does not improve query performance as expected when the distributed query engine is left unchanged. The scalability of distributed query processing is impaired by TCP overheads, switch contention due to uncoordinated communication, and load imbalances resulting from the inflexibility of the classic exchange operator model. This paper presents the blueprint for a distributed query engine that addresses these problems by considering both levels of networks holistically. It consists of two parts: First, hybrid parallelism that distinguishes local and distributed parallelism for better scalability in both the number of cores as well as servers. Second, a novel communication multiplexer tailored for analytical database workloads using remote direct memory access (RDMA) and low-latency network scheduling for high-speed communication with almost no CPU overhead. An extensive evaluation within the HyPer database system using the TPC-H benchmark shows that our holistic approach indeed enables high-speed query processing over high-speed networks.