Automated Cloud Provisioning and Management

点击翻译按钮获取中文摘要

Cloud computing harnesses massive resource capacity in data centers to support service applications in a scalable and cost-effective manner. The successful deployment of Cloud computing infrastructures and technologies in recent years is transforming the way IT business operates. However, the rapid growth in scale and complexity of today’s Cloud environments also raises critical concerns regarding the scalability and effectiveness of the underlying management systems, making Cloud provisioning and resource management an increasingly difficult challenge. To address this challenge, our long term objective is to design a scalable, efficient and reliable Cloud resource management framework that incurs minimal management overhead. Our general approach consists in (1) analyzing workload and resource characteristics, and (2) developing resource management schemes that leverage workload and resource characterizations. Our approach is motivated by the observation that Cloud resources (e.g., physical machines and network equipment) and workload (jobs and applications) often have heterogeneous yet characterizable behavior. For example, user-facing applications often have periodic (e.g., daily and seasonal) demand patterns and short-term trends. Similarly, batch (e.g., MapReduce) jobs with similar configurations are often executed in data centers on a routinely basis. Thus, by leveraging the knowledge gained from workload characterizations, it is possible to devise informed resource management schemes that attain higher application performance and resource utilization while reducing energy consumption. Specifically, the project aims at achieving the following 4 short-term objectives: (1) Characterizing workload in Cloud data centers is a relatively new research topic, especially for recent big-data applications such as MapReduce and Spark. Our recent work has shown that there is significant heterogeneity of both resources (e.g. capacity, performance, reliability and energy efficiency of physical hardware) and workloads (e.g. arrival rate, priority, resource usage, communication patterns and running time). Currently there is a lack of models for capturing such heterogeneity in a scalable yet accurate fashion. We intend to apply techniques from statistics and machine learning to address this challenge. (2) Despite extensive research on scheduling, the studies on resource-aware and performance (e.g. deadline)-aware scheduling for big-data analytics are rather limited. In this context, we plan to leverage resource and application profiles to make informed scheduling decisions with consideration to fairness, performance and resource efficiency. (3) Minimizing energy cost is a major concern of today’s Cloud providers. However, existing work on this topic often overlooks the heterogeneity in Cloud data centers. We intend to develop adaptive heterogeneity-aware energy management solutions to balance the trade-off between energy efficiency and application performance. (4) We also intend to extend our management framework to the context of geo-distributed clouds. In these settings, it is a major challenge to jointly provision service components (e.g., servers, load balancers, caches, proxies) and network resources to meet performance requirements in a dynamic manner. We intend to develop provisioning schemes that consider demand patterns to improve the performance and efficiency of geo-distributed services. The outcome of this project will significantly advance the state-of-the-art on Cloud resource provisioning and management. It will also have a profound impact on the development of industrial Cloud management systems. Finally, it will provide excellent opportunities for training highly qualified personal at the PhD and Master's levels.