BIGDATA: Collaborative Research: F: RDMA-Based Datacenter Networks for Online Big Data Applications

该项目解决了在线大数据(OLBD)应用程序在数据中心网络中实现极低延迟的挑战，这些应用程序是数据中心计算中的关键工作负载。远程直接内存访问(RDMA)是传统TCP的一种很有前途的替代方案，它显著减少了数据中心网络延迟约一个数量级。然而，RDMA的采用带来了两大挑战，因为RDMA在拥塞情况下存在性能脆弱性，并且RDMA会对典型的OLBD流量造成内存浪费或严重的程序员负担。该项目开发了两种新的网络技术--Blitz和RIMA--它们实现了可扩展的数据中心网络，实现了RDMA的低延迟优势，同时避免了其缺点(性能脆弱性、程序员负担和内存浪费)。Blitz通过分离边缘拥塞和网络内拥塞来解决性能脆弱性。闪电战使用接收方指导的拥塞控制(RDCC)来处理边缘拥塞，而不像以前的方法，发送方必须从往返时间和/或丢弃的分组间接推断发送速率。RDCC能够实现准确快速(一次往返时间内)的融合，从而实现更低的延迟和更高的吞吐量。闪电战通过将数据包转移到更长但不那么拥塞的路径来处理网络内的暂时性拥塞。远程间接内存访问(RIMA)通过实现反应性的按需内存分配来解决第二个挑战，而不是RDMA在最坏情况下的主动内存分配，这可以在不需要程序员努力的情况下最大限度地减少内存占用。Blitz和RIMA共同为OLBD应用程序实现了极低的数据中心网络延迟。该项目将广泛地让博士、硕士和本科生参与跨层研究活动。这些项目的成果将通过科学会议上的出版物广泛传播。

This project addresses the challenge of achieving extreme low latency in datacenter networks for Online Big Data (OLBD) applications which are critical workloads in datacenter computing. Remote Direct Memory Access (RDMA), which is a promising alternative to traditional TCP, significantly reduces datacenter network latencies by about an order-of-magnitude. However, RDMA adoption poses two major challenges as RDMA suffers from performance fragility under congestion, and RDMA incurs either wasted memory or significant programmer burden for typical OLBD traffic. This project develops two novel networking technologies -- Blitz and RIMA -- which enable scalable datacenter networks that achieve the low latency benefits of RDMA while avoiding its drawbacks (performance fragility, programmer burden, and wasted memory).Blitz addresses performance fragility by decoupling edge-congestion and in-network congestion. Blitz handles edge-congestion using receiver-directed congestion control (RDCC) unlike prior approaches where senders have to infer sending rates indirectly from round-trip-times and/or dropped packets. RDCC enables accurate and fast (within-one-round-trip-time) convergence, which leads to lower latency and higher throughput. Blitz handles transient in-network congestion by deflecting packets along longer yet less-congested paths.Remote Indirect Memory Access (RIMA) addresses the second challenge by enabling reactive, on-demand memory allocation as opposed to RDMAs proactive memory allocation for the worst case, which minimizes the memory footprint without programmer effort. Together, Blitz and RIMA enable extreme low datacenter network latency for OLBD applications. The project will extensively involve Ph.D, Masters, and undergraduate students in cross-layer research activities. Results from the projects will be broadly disseminated via publication in scientific conferences.