CRII: NeTS: High-Performance Scheduling Algorithms for Resource Management in Cloud Networks

点击翻译按钮获取中文摘要

Cloud computing has gained enormous momentum recently. By using cloud, clients no longer require to install and maintain their own infrastructure and can instead procure Virtual Machines (VMs) with specific configurations of CPU, memory, disk, and networking in the cloud. With the recent rise in demand for cloud services, it has become imperative to use the cloud resources efficiently to not only meet the demand, but also ensure scalability and cost efficiency. For instance, VMs can be potentially concentrated onto a smaller number of servers, and many of the unused servers can be turned off (to save energy) or utilized to increase the number of VMs that could be simultaneously supported by the system (to maximize throughput and reduce delay). A similar issue arises for assigning data flows between the servers to available paths in the network where the available bandwidth could be optimally utilized in order to improve congestion, delay, or energy saving.This project pursues three main objectives:(1) VM Scheduling: This project develops efficient low-complexity algorithms for VM packing, without explicit knowledge of the demand and without VM migrations, targeted for the dynamic setting in the cloud (i.e., VMs arrive/depart dynamically over time as opposed to being an static list). The technical approach is to extend packing to both space and time which is necessary for this dynamic setting.(2) Flow Scheduling: The applications hosted in the cloud have certain bandwidth requirements. Providing the bandwidth guarantees requires careful placement of data flows on available paths within the architecture. This project will study high-performance low-complexity algorithms for flow scheduling to balance the load and minimize the cost (e.g., delay, congestion, energy), without splitting and without re-routing the flows in the network.(3) Empirical Evaluation: To demonstrate the gains over the current datacenter heuristics, this project will test the algorithms on both synthesized and real cloud traffic traces, on various datacenter architectures, and under a wide range of workloads. The project will also use CloudLab both as a source for traces and as a platform for hosting experimental prototypes.Broader Impacts: The broader impacts of the project for the research community naturally arise from the wide prevalence of cloud and datacenters in modern Internet services. In particular, the PI envisions his algorithms and understanding of resource management in the cloud to further the state of the art in this field. This project has the potential to impact the IT industry by improving the scalability and cost efficiency of the cloud network architecture and to impact the society by reducing the energy usage of data centers and improving the user?s quality of experience. Industry impact and tech-transfer will happen through the industrial affiliates program of Data Science Institute at Columbia University. The PI encourages and actively seeks diversity within his research group. The inclusion of research ideas into classes and mentoring undergraduate/graduate students will ensure long-term percolation of the ideas into industry and academia.