CRII: NeTS: High-Performance Scheduling Algorithms for Resource Management in Cloud Networks

CRII：NetS：云网络资源管理的高性能调度算法

• 批准号: 1565774
• 负责人: Javad Ghaderi
• 金额: $ 17.49万
• 依托单位: Columbia University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-05-01 至 2019-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1565774&HistoricalAwards=false
• 关键词: CRII; NeTS; Performance; Scheduling; Algorithms

Cloud computing has gained enormous momentum recently. By using cloud, clients no longer require to install and maintain their own infrastructure and can instead procure Virtual Machines (VMs) with specific configurations of CPU, memory, disk, and networking in the cloud. With the recent rise in demand for cloud services, it has become imperative to use the cloud resources efficiently to not only meet the demand, but also ensure scalability and cost efficiency. For instance, VMs can be potentially concentrated onto a smaller number of servers, and many of the unused servers can be turned off (to save energy) or utilized to increase the number of VMs that could be simultaneously supported by the system (to maximize throughput and reduce delay). A similar issue arises for assigning data flows between the servers to available paths in the network where the available bandwidth could be optimally utilized in order to improve congestion, delay, or energy saving.This project pursues three main objectives:(1) VM Scheduling: This project develops efficient low-complexity algorithms for VM packing, without explicit knowledge of the demand and without VM migrations, targeted for the dynamic setting in the cloud (i.e., VMs arrive/depart dynamically over time as opposed to being an static list). The technical approach is to extend packing to both space and time which is necessary for this dynamic setting.(2) Flow Scheduling: The applications hosted in the cloud have certain bandwidth requirements. Providing the bandwidth guarantees requires careful placement of data flows on available paths within the architecture. This project will study high-performance low-complexity algorithms for flow scheduling to balance the load and minimize the cost (e.g., delay, congestion, energy), without splitting and without re-routing the flows in the network.(3) Empirical Evaluation: To demonstrate the gains over the current datacenter heuristics, this project will test the algorithms on both synthesized and real cloud traffic traces, on various datacenter architectures, and under a wide range of workloads. The project will also use CloudLab both as a source for traces and as a platform for hosting experimental prototypes.Broader Impacts: The broader impacts of the project for the research community naturally arise from the wide prevalence of cloud and datacenters in modern Internet services. In particular, the PI envisions his algorithms and understanding of resource management in the cloud to further the state of the art in this field. This project has the potential to impact the IT industry by improving the scalability and cost efficiency of the cloud network architecture and to impact the society by reducing the energy usage of data centers and improving the user?s quality of experience. Industry impact and tech-transfer will happen through the industrial affiliates program of Data Science Institute at Columbia University. The PI encourages and actively seeks diversity within his research group. The inclusion of research ideas into classes and mentoring undergraduate/graduate students will ensure long-term percolation of the ideas into industry and academia.

云计算最近获得了巨大的发展势头。通过使用云，客户不再需要安装和维护自己的基础架构，而是可以在云中购买具有特定CPU、内存、磁盘和网络配置的虚拟机（vm）。随着近年来对云服务需求的增加，高效利用云资源以满足需求的同时保证可扩展性和成本效益已成为当务之急。例如，vm可以潜在地集中在较少数量的服务器上，并且可以关闭许多未使用的服务器（以节省能源）或利用它们来增加系统可以同时支持的vm数量（以最大化吞吐量并减少延迟）。在将服务器之间的数据流分配给网络中的可用路径时，也会出现类似的问题，在这些路径中，可用带宽可以得到最佳利用，以改善拥塞、延迟或节能。该项目追求三个主要目标：(1)虚拟机调度：该项目为虚拟机打包开发高效的低复杂度算法，没有明确的需求知识，没有虚拟机迁移，针对云中的动态设置（即，虚拟机随着时间的推移动态到达/离开，而不是静态列表）。技术方法是将包装扩展到空间和时间，这对于这种动态设置是必要的。(2)流量调度：云上托管的应用对带宽有一定的要求。提供带宽保证需要在架构内的可用路径上仔细放置数据流。本项目将研究流调度的高性能低复杂度算法，以平衡负载和最小化成本（例如，延迟，拥塞，能源），而不分裂和不重新路由网络中的流。(3)经验评估：为了展示当前数据中心启发式算法的收益，本项目将在综合和真实云流量轨迹、各种数据中心架构和广泛的工作负载下测试算法。该项目还将使用CloudLab作为跟踪源和托管实验原型的平台。更广泛的影响：该项目对研究界的更广泛的影响自然来自于现代互联网服务中云和数据中心的广泛流行。特别是，PI设想他的算法和对云中资源管理的理解，以进一步提高该领域的技术水平。该项目有潜力通过提高云网络架构的可扩展性和成本效率来影响IT行业，并通过减少数据中心的能源使用和改善用户体验来影响社会。经验的质量。行业影响和技术转移将通过哥伦比亚大学数据科学研究所的工业附属项目实现。PI鼓励并积极寻求他的研究小组的多样性。将研究想法纳入课堂并指导本科生/研究生，将确保这些想法长期渗透到工业界和学术界。

项目成果

An Improved Bound for Minimizing the Total Weighted Completion Time of Coflows in Datacenters

• DOI: 10.1109/tnet.2018.2845852
• 发表时间: 2017-04
• 期刊: IEEE/ACM Transactions on Networking
• 作者: Mehrnoosh Shafiee;Javad Ghaderi