Direct Telemetry Access
Direct Telemetry Access
复制标题
DOI：
10.1145/3603269.3604827
复制
发表时间：
2022-02
期刊：
影响因子：
--
通讯作者：
Jonatan Langlet;Ran Ben Basat;G. Oliaro;M. Mitzenmacher;Minlan Yu;G. Antichi
中科院分区：
文献类型：
--
作者：
Jonatan Langlet;Ran Ben Basat;G. Oliaro;M. Mitzenmacher;Minlan Yu;G. Antichi
细粒度网络遥测正在成为现代数据中心的标准，并且是拥塞控制、负载平衡和高级故障排除等基本应用的基础。随着网络规模的增加和遥测变得更加细粒度，需要从交换机报告到收集器以实现网络范围视图的数据量急剧增长。因此，逐步难以扩展数据收集系统。我们推出了直接远程监控访问（DTA），这是一种经过优化的解决方案，可将每秒来自交换机的数亿份报告聚合并移动到采集器内存中的可查询数据结构中。DTA重量轻，能够大大减少收集器的管理费用。DTA是建立在RDMA之上的，我们提出了新颖的和富有表现力的报告原语，可以轻松地与现有的最先进的遥测机制，如INT或马普尔集成。我们表明，DTA显着提高遥测收集率。例如，当与INT一起使用时，它可以通过单个服务器每秒收集和聚合超过4亿份报告，比Atomic MultiLog提高了16倍。
Fine-grained network telemetry is becoming a modern datacenter standard and is the basis of essential applications such as congestion control, load balancing, and advanced troubleshooting. As network size increases and telemetry gets more fine-grained, there is a tremendous growth in the amount of data needed to be reported from switches to collectors to enable network-wide view. As a consequence, it is progressively hard to scale data collection systems. We introduce Direct Telemetry Access (DTA), a solution optimized for aggregating and moving hundreds of millions of reports per second from switches into queryable data structures in collectors' memory. DTA is lightweight and it is able to greatly reduce overheads at collectors. DTA is built on top of RDMA, and we propose novel and expressive reporting primitives to allow easy integration with existing state-of-the-art telemetry mechanisms such as INT or Marple. We show that DTA significantly improves telemetry collection rates. For example, when used with INT, it can collect and aggregate over 400M reports per second with a single server, improving over the Atomic MultiLog by up to 16x.