Automated Cloud Provisioning and Management

云计算利用数据中心的大量资源容量，以可扩展和经济高效的方式支持服务应用程序。近年来，云计算基础设施和技术的成功部署正在改变IT业务的运作方式。然而，当今云环境的规模和复杂性的快速增长也引起了对底层管理系统的可伸缩性和有效性的关键关注，使云供应和资源管理成为越来越困难的挑战。为了应对这一挑战，我们的长期目标是设计一个可伸缩、高效和可靠的云资源管理框架，使管理开销最小化。我们的一般方法包括：(1)分析工作负载和资源特征，以及(2)开发利用工作负载和资源特征的资源管理方案。我们的方法的动机是观察到云资源（例如，物理机器和网络设备）和工作负载（作业和应用程序）通常具有异构但可表征的行为。例如，面向用户的应用程序通常具有周期性（例如，每日和季节性）需求模式和短期趋势。类似地，具有类似配置的批处理（例如，MapReduce）作业通常在数据中心例行执行。因此，通过利用从工作负载特征中获得的知识，可以设计明智的资源管理方案，从而在降低能耗的同时获得更高的应用程序性能和资源利用率。具体而言，该项目旨在实现以下4个短期目标：(1)表征云数据中心的工作负载是一个相对较新的研究课题，特别是对于最近的大数据应用，如MapReduce和Spark。我们最近的工作表明，资源（例如物理硬件的容量、性能、可靠性和能源效率）和工作负载（例如到达率、优先级、资源使用、通信模式和运行时间）都存在显著的异质性。目前，缺乏以可扩展但准确的方式捕获这种异质性的模型。我们打算应用统计学和机器学习技术来应对这一挑战。(2)尽管对调度进行了广泛的研究，但对大数据分析中资源感知和性能感知（如截止日期）调度的研究相当有限。在这种情况下，我们计划利用资源和应用程序配置文件，在考虑公平性、性能和资源效率的情况下做出明智的调度决策。(3)最小化能源成本是当今云提供商关注的主要问题。然而，关于这个主题的现有工作往往忽略了云数据中心的异构性。我们打算开发自适应的异构感知能源管理解决方案，以平衡能源效率和应用程序性能之间的权衡。(4)我们还打算将我们的管理框架扩展到地理分布式云环境。在这些设置中，联合提供服务组件（例如，服务器、负载平衡器、缓存、代理）和网络资源以动态方式满足性能需求是一个主要挑战。我们打算开发考虑需求模式的供应方案，以提高地理分布式服务的性能和效率。该项目的成果将显著推动云资源供应和管理的发展。它也将对工业云管理系统的发展产生深远的影响。最后，它将为培养博士和硕士水平的高素质人才提供极好的机会。

Cloud computing harnesses massive resource capacity in data centers to support service applications in a scalable and cost-effective manner. The successful deployment of Cloud computing infrastructures and technologies in recent years is transforming the way IT business operates. However, the rapid growth in scale and complexity of today’s Cloud environments also raises critical concerns regarding the scalability and effectiveness of the underlying management systems, making Cloud provisioning and resource management an increasingly difficult challenge. To address this challenge, our long term objective is to design a scalable, efficient and reliable Cloud resource management framework that incurs minimal management overhead. Our general approach consists in (1) analyzing workload and resource characteristics, and (2) developing resource management schemes that leverage workload and resource characterizations. Our approach is motivated by the observation that Cloud resources (e.g., physical machines and network equipment) and workload (jobs and applications) often have heterogeneous yet characterizable behavior. For example, user-facing applications often have periodic (e.g., daily and seasonal) demand patterns and short-term trends. Similarly, batch (e.g., MapReduce) jobs with similar configurations are often executed in data centers on a routinely basis. Thus, by leveraging the knowledge gained from workload characterizations, it is possible to devise informed resource management schemes that attain higher application performance and resource utilization while reducing energy consumption. Specifically, the project aims at achieving the following 4 short-term objectives:(1) Characterizing workload in Cloud data centers is a relatively new research topic, especially for recent big-data applications such as MapReduce and Spark. Our recent work has shown that there is significant heterogeneity of both resources (e.g. capacity, performance, reliability and energy efficiency of physical hardware) and workloads (e.g. arrival rate, priority, resource usage, communication patterns and running time). Currently there is a lack of models for capturing such heterogeneity in a scalable yet accurate fashion. We intend to apply techniques from statistics and machine learning to address this challenge.(2) Despite extensive research on scheduling, the studies on resource-aware and performance (e.g. deadline)-aware scheduling for big-data analytics are rather limited. In this context, we plan to leverage resource and application profiles to make informed scheduling decisions with consideration to fairness, performance and resource efficiency. (3) Minimizing energy cost is a major concern of today’s Cloud providers. However, existing work on this topic often overlooks the heterogeneity in Cloud data centers. We intend to develop adaptive heterogeneity-aware energy management solutions to balance the trade-off between energy efficiency and application performance. (4) We also intend to extend our management framework to the context of geo-distributed clouds. In these settings, it is a major challenge to jointly provision service components (e.g., servers, load balancers, caches, proxies) and network resources to meet performance requirements in a dynamic manner. We intend to develop provisioning schemes that consider demand patterns to improve the performance and efficiency of geo-distributed services.The outcome of this project will significantly advance the state-of-the-art on Cloud resource provisioning and management. It will also have a profound impact on the development of industrial Cloud management systems. Finally, it will provide excellent opportunities for training highly qualified personal at the PhD and Master's levels.