SHF: Small: Towards Cost-Efficient Guaranteed Performance Multicast in Fat-Tree Data Center Networks

由数以万计的服务器组成的海量现代数据中心，如微软的Azure平台、谷歌的App Engine和亚马逊的EC2平台，已经出现，形成了各种强大的分布式计算框架的骨干。与此同时，许多公司正在将电子商务、科学计算和社交网络等服务转移到云上，因为它能够提供可扩展和弹性的计算和存储服务。在这种大规模的分布式计算框架中，往往需要在存储在一个数据中心的数万台服务器上的海量数据集之间进行高效的通信。连接不同服务器的数据中心网络(DCN)将成为系统的瓶颈，其性能对数据中心的成功运营至关重要。另一方面，数据中心托管的许多在线应用和后端基础设施计算需要从一个服务器到一组服务器的一对多或多播通信。本研究旨在研究在构建具有成本效益和性能保证的组播数据中心网络时所面临的基本和具有挑战性的问题。随着云计算正在渗透到社会的方方面面，这项研究将对社会产生深刻的影响，并帮助改变世界。本研究的目的是通过探索数据中心中一些独特的新特性和技术来设计具有成本效益和性能保证的组播胖树数据中心网络(DCNS)。该项目结合了理论分析、算法设计、网络优化、仿真和原型技术，提供了一个全面的工作解决方案，能够在胖树DCNS中实现高性能的组播。更具体地说，研究重点集中在以下紧密耦合的问题上：(1)通过探索数据中心的服务器冗余和链路超额订阅，以成本效率的方式提供胖树DCN来部署有保证的带宽组播；(2)利用OpenFlow框架开发实用的组播调度算法，以确保数据中心流量的负载平衡和网络的高效利用；(3)利用虚拟机技术提供针对特定应用需求的具有区分带宽保证的组播；(4)通过在网络原型中广泛的模拟和实施所提出的方案，对所提出的方案进行全面的性能评估。本研究希望对高性能组播胖树DCNS的基本设计原则产生一定的影响。这项研究的结果有可能提升目前托管在数据中心的云计算应用的性能，并促进未来依赖组通信和需求可预测高带宽的应用采用云。一个项目的目标是培养研究生，并促进女性工科学生的参与。该项目的重要成果将通过会议、期刊和网站访问的方式向研究界传播。

Massive modern data centers consisting of tens of thousands of servers, such as Microsoft's Azure platform, Google's App engine, and Amazon's EC2 platform, have emerged to form the backbone of a variety of powerful distributed computing frameworks. Meanwhile, many companies are moving their services such as e-commerce, scientific computing and social networking to the cloud, due to its ability to offer scalable and elastic computing and storage services. In such large-scale distributed computing frameworks, efficient communication is often required among huge datasets stored in tens of thousands of servers across a data center. The data center network (DCN) that connects different servers would become the bottleneck of the system, and its performance is essential to the successful operation of a data center. On the other hand, many online applications and back-end infrastructural computations hosted by data centers require one-to-many or multicast communication from a server to a group of servers. This research aims to investigate the fundamental and challenging issues faced in building cost-efficient multicast data center networks with guaranteed performance. As cloud computing is penetrating into all aspects of society, this research will have a profound impact on society and help change the world. The objective of this research is to design cost-efficient multicast fat-tree data center networks (DCNs) with guaranteed performance through exploring some unique novel features and techniques in data centers. The project combines theoretical analysis, algorithm design, network optimization, simulation, and prototyping techniques to provide a comprehensive working solution that enables high performance multicast in fat-tree DCNs. More specifically, the research focuses on following closely coupled issues: (1) cost-efficient provisioning of fat-tree DCNs to deploy guaranteed-bandwidth multicast by exploring server redundancy and link oversubscription in data centers; (2) leveraging the OpenFlow framework to develop practical multicast scheduling algorithms that ensure traffic load balance and efficient network utilization under volatile data center traffic; (3) employing virtual machine technology to offer multicast with differentiated bandwidth guarantees tailored to application-specific demand; (4) conducting a comprehensive performance evaluation through extensive simulations and implementation of proposed schemes in a network prototype. This research hopes to impact fundamental design principles of high performance multicast fat-tree DCNs. The outcome of this research has the potential to boost the performance of cloud computing applications currently hosted in data centers, and to facilitate cloud adoption for future applications that rely on group communication and demand predictable high bandwidth. A project goal is to train graduate students and promote the participation of female engineering students. The important findings of this project are to be disseminated to the research community by way of conferences, journals and web site access.