Republic: Data Multicast Meets Hybrid Rack-Level Interconnections in Data Center
Republic: Data Multicast Meets Hybrid Rack-Level Interconnections in Data Center
复制标题
DOI：
10.1109/icnp.2018.00018
复制
发表时间：
2018-09
期刊：
影响因子：
--
通讯作者：
Xiaoye Sun;Yiting Xia;Simbarashe Dzinamarira;Xin Sunny Huang;Dingming Wu;T. Ng
中科院分区：
文献类型：
--
作者：
Xiaoye Sun;Yiting Xia;Simbarashe Dzinamarira;Xin Sunny Huang;Dingming Wu;T. Ng
数据组播是分布式大数据处理中一种重要的数据传输方式。然而，由于缺乏网络和系统级的支持，数据处理依赖于基于单播的应用层组播。近年来，对使用各种新兴电路交换技术来构建具有混合分组-电路交换机架级互连的数据中心的兴趣激增，即，混合数据中心。这些物理层的创新从根本上改变了机架间的通信能力，特别是组播通信能力。我们提出共和国，一个完整的系统，解决了具有挑战性的问题，实现高性能的数据组播在混合数据中心。Republic将底层网络复杂性抽象为数据组播服务，为请求数据组播的数据中心应用提供统一的Republic API。Republic在混合数据中心测试平台中实施和部署。测试床评估表明，Republic可以将Apache Spark机器学习应用程序中的数据多播提高4.0倍。
Data multicast is a crucial data transfer pattern in distributed big-data processing. However, due to the lack of network and system level support, data processing relies on unicast-based application layer multicast. In recent years, there has been a surge in interest in using various emerging circuit switching technologies to build data centers having hybrid packet-circuit switched rack-level interconnections, i.e., hybrid data centers. These physical layer innovations fundamentally change the inter-rack communication capability, especially the capability of multicast communication. We propose Republic, a complete system that addresses the challenging issues in achieving high-performance data multicast in hybrid data centers. Republic abstracts the underlying network complexity as a data multicast service and provides a unified Republic API for data center applications requesting data multicast. Republic is implemented and deployed in a hybrid data center testbed. Testbed evaluation shows that Republic can improve data multicast in Apache Spark machine learning applications by as much as 4.0x.