CRII: NeTS: High-Performance Scheduling Algorithms for Resource Management in Cloud Networks

云计算最近获得了巨大的发展势头。通过使用云，客户不再需要安装和维护自己的基础设施，而是可以在云中购买具有特定CPU，内存，磁盘和网络配置的虚拟机（VM）。随着最近对云服务需求的增加，有效地使用云资源不仅要满足需求，还要确保可扩展性和成本效益。例如，VM可以潜在地集中到较少数量的服务器上，并且许多未使用的服务器可以被关闭（以节省能量）或用于增加可以由系统同时支持的VM的数量（以最大化吞吐量并减少延迟）。同样的问题也出现在将服务器之间的数据流分配到网络中的可用路径上，在这些路径上可以最佳地利用可用带宽，以改善拥塞、延迟或节能。本项目追求三个主要目标：（1）VM调度：该项目开发了有效的低复杂度算法，用于VM打包，无需明确了解需求，也无需VM迁移，以云中的动态设置为目标（即，VM随时间动态地到达/离开，而不是静态列表）。技术方法是将包装扩展到空间和时间，这对于这种动态设置是必要的。(2)流调度：托管在云中的应用程序有一定的带宽要求。提供带宽保证需要在架构内的可用路径上仔细放置数据流。本项目将研究用于流调度的高性能低复杂度算法，以平衡负载并最小化成本（例如，延迟、拥塞、能量），而不分裂并且不重新路由网络中的流。(3)经验评价：为了展示当前数据中心的优势，该项目将在合成和真实的云流量跟踪、各种数据中心架构和各种工作负载下测试算法。该项目还将使用CloudLab作为跟踪源和托管实验原型的平台。更广泛的影响：该项目对研究社区的更广泛影响自然来自现代互联网服务中云和数据中心的广泛普及。特别是，PI设想他的算法和对云中资源管理的理解，以进一步发展该领域的最新技术。该项目有可能通过提高云网络架构的可扩展性和成本效率来影响IT行业，并通过减少数据中心的能源使用和改善用户的安全性来影响社会。的经验质量。行业影响和技术转移将通过哥伦比亚大学数据科学研究所的行业附属计划进行。PI鼓励并积极寻求他的研究小组的多样性。将研究思想纳入课堂并指导本科生/研究生，将确保这些思想长期渗透到工业界和学术界。

Cloud computing has gained enormous momentum recently. By using cloud, clients no longer require to install and maintain their own infrastructure and can instead procure Virtual Machines (VMs) with specific configurations of CPU, memory, disk, and networking in the cloud. With the recent rise in demand for cloud services, it has become imperative to use the cloud resources efficiently to not only meet the demand, but also ensure scalability and cost efficiency. For instance, VMs can be potentially concentrated onto a smaller number of servers, and many of the unused servers can be turned off (to save energy) or utilized to increase the number of VMs that could be simultaneously supported by the system (to maximize throughput and reduce delay). A similar issue arises for assigning data flows between the servers to available paths in the network where the available bandwidth could be optimally utilized in order to improve congestion, delay, or energy saving.This project pursues three main objectives:(1) VM Scheduling: This project develops efficient low-complexity algorithms for VM packing, without explicit knowledge of the demand and without VM migrations, targeted for the dynamic setting in the cloud (i.e., VMs arrive/depart dynamically over time as opposed to being an static list). The technical approach is to extend packing to both space and time which is necessary for this dynamic setting.(2) Flow Scheduling: The applications hosted in the cloud have certain bandwidth requirements. Providing the bandwidth guarantees requires careful placement of data flows on available paths within the architecture. This project will study high-performance low-complexity algorithms for flow scheduling to balance the load and minimize the cost (e.g., delay, congestion, energy), without splitting and without re-routing the flows in the network.(3) Empirical Evaluation: To demonstrate the gains over the current datacenter heuristics, this project will test the algorithms on both synthesized and real cloud traffic traces, on various datacenter architectures, and under a wide range of workloads. The project will also use CloudLab both as a source for traces and as a platform for hosting experimental prototypes.Broader Impacts: The broader impacts of the project for the research community naturally arise from the wide prevalence of cloud and datacenters in modern Internet services. In particular, the PI envisions his algorithms and understanding of resource management in the cloud to further the state of the art in this field. This project has the potential to impact the IT industry by improving the scalability and cost efficiency of the cloud network architecture and to impact the society by reducing the energy usage of data centers and improving the user?s quality of experience. Industry impact and tech-transfer will happen through the industrial affiliates program of Data Science Institute at Columbia University. The PI encourages and actively seeks diversity within his research group. The inclusion of research ideas into classes and mentoring undergraduate/graduate students will ensure long-term percolation of the ideas into industry and academia.